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ACCOUNTABILITY AND RESULTS IN FEDERAL 

BUDGETING 


TUESDAY, JUNE 14, 2005 

U.S. Senate, 

Subcommittee on Federal Financial Management, 

Government Information, and International Security, 

OF THE Committee on Homeland Security 
AND Governmental Affairs, 

Washington, DC. 

The Subcommittee met, pursuant to notice, at 2 p.m., in room 
562, Dirksen Senate Office Building, Hon. Tom Coburn, Chairman 
of the Subcommittee, presiding. 

Present: Senators Coburn, Carper, and Lautenberg. 

OPENING STATEMENT OF SENATOR COBURN 

Senator Coburn. The Subcommittee will come to order. This 
hearing today is on Accountability and Results in Federal Budg- 
eting. The Federal Government is expected to spend $2.5 trillion in 
the next fiscal year, which comes to $22,000 per family in the 
United States, a significant sum. While the first hearing of this 
Subcommittee focused broadly on the President’s Management 
Agenda, also known as PMA, today, we will more specifically dis- 
cuss efforts to increase accountability in Federal budgeting — ac- 
countability as seen through transparency, on-time reports, evalua- 
tion, and assessment. 

As part of the PMA, 0MB released in 2003 the Program Assess- 
ment Rating Tool, or PART. Used to evaluate the design, goals, and 
performance of Federal programs, PART seeks to find ways to ulti- 
mately improve overall performance through the format of a basic 
questionnaire and evaluation of that. Thus far, PART has been 
used to evaluate 607 Federal Government programs, roughly 60 
percent of the Federal budget. Of these 607 programs, 15 percent 
have been rated effective, less than 90; 26 percent moderately effec- 
tive; 4 percent ineffective; 29 percent could not demonstrate re- 
sults, and 26 percent, adequate. 

That last number that I gave you, the 29 percent that could not 
demonstrate results, it was almost a third of the Federal Govern- 
ment’s programs. 

We will also discuss today the latest iteration of OMB’s score- 
card, which is a set of quarterly grades for each Federal agency. 
Ratings of red, yellow, and green are given to each agency for each 
of five initiatives: Human capital, competitive sourcing, financial 
performance, e-government, and budget and performance integra- 
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tion. Ratings are given for both current status and progress in im- 
plementing the PMA, the President’s Management Agenda. 

When the first scorecard was issued in June 2002, only 4 agen- 
cies received yellow current status ratings for their budget perform- 
ance and integration, while the remaining 22 agencies all received 
red ratings. In the latest scorecard, 6 agencies have red ratings, 12 
have yellow ratings, and 8 have green current ratings for budget 
performance and integration. While the latest of these scores are 
encouraging, they also demonstrate that the Federal Government 
has a lot to do when it comes to managing the way it spends the 
taxpayers’ money. 

As part of the President’s budget proposal for fiscal year 2006, 
0MB released a list of roughly 150 discretionary programs for 
which it proposed either reduced funding or complete elimination. 
The termination of many of these programs has been proposed be- 
fore. For example, the 0MB proposed the termination of the Ad- 
vanced Technology Program four separate times. The termination 
of earmarks for the Centers for Disease Control and Prevention 
was proposed not only in fiscal year 2006, but in 2002, 2003, 2004, 
and 2005, as well. The termination of the program for Community 
Technology Centers has been proposed six separate times. 

It is absolutely stunning that we continue to fund programs that 
time after time, year after year, fail to produce positive results or 
measurable results. I hope we will hear substantive proposals 
today to either terminate or measurably reform these programs. 

We are pleased to have with us today representatives from both 
government and academia. On our first panel, the Hon. David 
Walker, Comptroller General of the Government Accountability Of- 
fice, and the Hon. Clay Johnson, III, the Deputy Director for Man- 
agement at the Office of Management and Budget, will give us 
their perspective on efforts to increase accountability and ulti- 
mately improve results in Federal budgeting. 

We are also privileged to have on our second panel Eileen Nor- 
cross, a Research Fellow with George Mason University and The 
Mercatus Center, and Beryl Radin, a professor of Government and 
Public Administration at the University of Baltimore. They will 
both give us a helpful non-governmental perspective on the effect 
of government accountability efforts. 

Our Ranking Member, Senator Carper, will be here in a moment. 
We will ask him for his opening statement at that time. At the 
present time, I would like to introduce our witnesses. Our first wit- 
ness is the Hon. David Walker, Comptroller General of the United 
States. Mr. Walker began his 15-year term as the Nation’s chief ac- 
countability officer and was appointed in 1998 as the head of the 
then-General Accounting Office, now referred to as the Government 
Accountability Office. Through his role as Comptroller General, Mr. 
Walker oversees GAO’s work to improve the performance and ac- 
countability of the Federal Government, including measures to im- 
prove the efficient and effective use of taxpayer dollars. 

Our second witness on the first panel today is the Hon. Clay 
Johnson, III, Deputy Director of Management for the Office of 
Management and Budget. In this role, Mr. Johnson provides gov- 
ernment-wide leadership to the Executive Branch agencies to im- 
prove the agency and program performance. Prior to this position. 
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Mr. Johnson served as Assistant to the President for Presidential 
personnel and as the Executive Director of the Bush-Cheney transi- 
tion team. 

I would like to thank both Mr. Walker and Mr. Johnson for being 
here. They have been here before and will be here again. We are 
very pleased with your work, and I would note that your submitted 
statements will be made a part of the record and you will each be 
recognized for 5 minutes. 

Mr. Walker. 

TESTIMONY OF HON. DAVID M. WALKER, i COMPTROLLER GEN- 
ERAL OF THE UNITED STATES, U.S. GOVERNMENT ACCOUNT- 
ABILITY OFFICE 

Mr. Walker. Thank you, Mr. Chairman. It is a pleasure to be 
back before this Subcommittee. I appreciate your commitment to 
improving government performance and ensuring accountability for 
the American people. I thank you for including my entire state- 
ment into the record. I will now move to summarize it. 

I am pleased to come here before the Subcommittee today to talk 
about the concept of performance budgeting in general and the Of- 
fice of Management and Budget’s Program Assessment Rating Tool, 
or PART, in particular. As you know, Mr. Chairman, our Nation is 
currently on an unsustainable fiscal path. I have two graphics that 
I would like to refamiliarize you and the other Members and key 
staff with. 2 

Both are based upon GAO’s long-range budget simulations. This 
first one is based on CBO’s 10-year baseline projections and the re- 
quirements that CBO, by law, must comply with. Those require- 
ments, among other things, include: (1) no new laws will be passed 
in the next 10 years, (2) all expiring tax cuts will, in fact, sunset, 
and (3) discretionary spending will grow by the rate of inflation 
during the next 10 years, and (4), that the alternative minimum 
tax will not be fixed. 

Mr. Chairman, I have asked individuals in every speech that I 
have given in the last 2 months whether or not they believe any 
of those four assumptions are true, and so far, I have less than ten 
out of several thousand that believe that those assumptions are 
reasonable. Unfortunately, this is the basis Congress is using to 
make decisions. 

The next chart demonstrates what an alternative scenario would 
look like if all expiring tax cuts are made permanent and if discre- 
tionary spending grows by the rate of the economy during the en- 
tire period. It is clearly a very dramatic and unacceptable outcome. 

As a result, it is critically important that a fundamental reexam- 
ination of major spending and tax policies and priorities be under- 
taken in order to recapture our fiscal flexibility for the future and 
address key social, economic, and security changes and challenges 
in the 21st Century. Clearly, performance budgeting holds promise 
as part of a fundamental reexamination of the basis of the Federal 
Government. Existing performance budgeting efforts, such as the 
Government Performance and Results Act (GPRA), and PART, (or 


^The prepared statement of Mr. Walker appears in the Appendix on page 33. 

2 The graphics referred to in Mr. Walkers prepared statement appear in the Appendix on 
pages 39 and 40 respectively. 
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the Program Assessment Rating Tool), can provide a foundation for 
a baseline review of existing Federal policies, programs, functions, 
and activities. 

As I testified before this Subcommittee in April, the President’s 
Management Agenda and its related initiatives, including PART, 
demonstrate the Administration’s commitment to improving not 
only Federal financial management but also overall management 
while enhancing government performance. However, it is not clear 
that PART has had any significant impact on Congressional au- 
thorization, appropriations, and oversight activities to date. 

In our view, there are three key factors that we believe are crit- 
ical to sustaining successful performance budgeting over time. One, 
we have to build a supply of credible performance information. 
Two, we have to encourage demand for that information and its use 
in Congressional processes by garnering stakeholder buy-in. And 
three, we need to take a comprehensive and cross-cutting approach 
to assessing related programs and policies which must be not just 
vertical, but horizontal, and must consider not just spending, but 
also tax policies and preferences. 

The Federal Government is in a period of profound transition. 
We face an array of changes, challenges, and opportunities to en- 
hance performance and assure accountability. Much is at stake in 
the development of a collaborative performance budgeting process. 
This is an opportune time for the Executive Branch and the Con- 
gress to consider and discuss how agencies and committees can 
best take advantage of and leverage the new information and per- 
spectives coming from the reform agenda currently underway. 
Some program improvements can come solely through Executive 
Branch action, but for PART to meet its intended goal, there must 
be greater buy-in by the Congress, which to date has not been 
forthcoming. 

I would be happy to answer any questions that you and the other 
Members of the Subcommittee may have, Mr. Chairman. Thank 
you. 

Senator Coburn. Thank you, Mr. Walker. Mr. Johnson. 

TESTIMONY OF HON. CLAY JOHNSON, HI,i DEPUTY DIRECTOR 
FOR MANAGEMENT, OFFICE OF MANAGEMENT AND BUDGET 

Mr. Johnson. Mr. Chairman, Senator Lautenberg, thank you. I 
contend that agencies are better managed today, and they are more 
focused on results than ever before. But we are not as focused on 
results as we can and need to be. 

Agencies are assessing program performance with the PART in- 
strument and using this information to inform management and 
budget actions. The Administration is proposing Sunset and Re- 
sults Commissions legislation to involve Congress more directly in 
holding agencies accountable for results. 

Some say the PART assessments have not had any impact on au- 
thorizations, appropriations, or oversight. To that, I say it has had 
some impact, but clearly, we can do a better job convincing Con- 
gress of the usefulness of performance information. I point out that 
this effort was designed to span 5 years. Only this past year did 


^The prepared statement of Mr. Johnson appears in the Appendix on page 56. 
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we assess programs accounting for more than 50 percent of the 
budget. We are only in the fourth year of a 5-year effort, and there 
is more attention being paid this year than last. 

Some would also say that PART assumes that each program has 
one goal and that all programs are alike. To that, I say all pro- 
grams are alike in that they should be able to account for what tax- 
payers are getting for their money. Most importantly, I point out, 
though, that the PART asks unique questions for different kinds of 
programs. They ask unique questions for competitive grant pro- 
grams, block grant programs, regulatory-based programs, capital 
asset programs, credit programs, and research and development 
programs. 

Some say that good performance data is hard to come by. I agree, 
but shame on us if we are not always looking for the best way, no 
matter how imperfect, to measure what our programs do, and what 
outcomes they achieve. 

And some say that the Results and Sunset Commissions that we 
are about to propose are ways to get rid of programs we do not like. 
To that, I say we want programs to work. The history of Sunset 
Commissions and similar programs is that they are much more apt 
to drive program improvement than they are to result in program 
elimination. 

We all want to get more for the money we spend. The biggest op- 
portunity is to get programs to work better. Yes, we debate in the 
preparation of our budgets and in appropriations legislation on 
whether to eliminate $5 or $10 billion of programs or how else to 
spend it, but this pales in comparison to the $20-plus billion associ- 
ated with every 1 percent improvement in performance. The most 
significant opportunity we have is to drive better program perform- 
ance. 

We believe that the PART now, and the soon to be proposed Sun- 
set and Results Commissions, help us achieve these savings for the 
taxpayers. They help us focus on results. Thank you. 

Senator Coburn. Thank you, Mr. Johnson. 

Senator Lautenberg, would you like to make an opening state- 
ment. 


OPENING STATEMENT OF SENATOR LAUTENBERG 

Senator Lautenberg. Thank you very much, Mr. Chairman. For- 
give my tardiness. I am sorry that I missed the Subcommittee 
meeting that you had, but it was impossible to be back here from 
distant travel. 

I do thank you for convening this hearing, and thanks also for 
giving us an opportunity to discuss the issue. Mr. Chairman, I 
think everyone knows I came out of the business community, and 
before I arrived in the Senate, when I left that company 23 years 
ago, we had 16,000 employees and the cardinal principle was to 
make sure everybody carried their share and we held them ac- 
countable. As a result of that kind of supervision, we had a really 
successful career as a company and it continued on way past my 
chairmanship, so apparently I got them off on a good start. 

So I applaud the President’s desire to hold the government pro- 
grams and employees accountable for their performance. The first 
step in accountability is an ability to measure performance, and 
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that is what the PART program is about. Measuring performance 
is like any other tool in business or government. It can be effective, 
but only if it is used properly, and that, Mr. Chairman, is my con- 
cern about the PART program. 

We have to ask the right questions, measure the right things. If 
we don’t, we will never get good results. So we have to make sure 
that the questions are directed in such a way that there is an ob- 
jective review. Programs should be judged based on how well they 
achieve the goals set for them and whether or not they have any 
political pressure to present the results of their review. 

Since this hearing is about a performance-based budget, there is 
one thing that I would like to mention in passing, and that is in 
business, there is a measure of success that we don’t have available 
to us here and that is the financial result, financial bottom-line. 
But government just doesn’t and can’t really do it the same way. 
Our definition of success is much more complicated. When a pro- 
gram doesn’t perform, we have to ask why not. 

For instance, the Manufacturing Extension Partnership Program 
lost points on its PART evaluation because in spite of its successes 
in leveraging financial support, it serves only a small percentage 
of small manufacturers each year, and that evaluation was used to 
cut the funding for the program. It doesn’t make sense. If the pro- 
gram is working, it doesn’t have enough resources to make a big 
enough impact, we should decide on whether or not we are going 
to increase it or get rid of the program. 

In short, Mr. Chairman, we should demand accountability for the 
Federal Government to make sure they accomplish the goals that 
we set for them, but we have to examine the outcomes of these pro- 
grams honestly, fearlessly, and without any bias in the way we see 
the programs other than in their efficiency and their results. 

So I thank you, Mr. Chairman, for doing this. 

Senator Coburn. Thank you. I would clarify for the record, in 
both Mr. Walker’s testimony and Mr. Johnson’s testimony, I believe 
there is reference to the fact that when programs fall low on the 
PART assessment, that sometimes it is because not enough money 
has been given. So I think they recognize that weakness and they 
have testified that way. 

I want to go back to your chart, because whether we have a Re- 
sults or a Sunset Commission or whether we have a PART evalua- 
tion, right now, we are sitting at about 19.6 percent of our GDP, 
the government consuming it, according to what you have right 
there, and estimated to grow to approximately 40 percent by the 
year 2040. 

We have a tool now that we are using — that we are attempting 
to use. We are not effectively using it in a lot of ways because a 
lot of the agencies aren’t responding with the management exper- 
tise based on the measurement tool that is going to be used there. 
What is the plan to move those numbers down through PART, 
through results, through Sunset Commission, so even though we 
have this long run of mandatory spending, how in the world are 
we going to achieve that area that we can achieve and how are we 
going to be able to implement this? 

We are in the third year, fourth year of this. What do we see? 
Are we seeing improvements? The criticisms of the PART system. 
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are they legitimate? Is it resistance to just being measured, or are 
there some legitimate criticisms to the PART system, either one of 
you that might want to respond to that. 

Mr. Walker. I will start, Mr. Chairman, by refamiliarizing you, 
and I know you have read this at least once, the booklet that we 
put out 

Senator Coburn. Twice. 

Mr. Walker. Thank you very much, Mr. Chairman. Specifically, 
I am referring to the booklet that we put out on February 16 which 
lays out the business case pretty strongly that we are on an impru- 
dent and unsustainable fiscal path. As a result we are going to 
have to look at the basis of the Federal Government, both as it re- 
lates to spending and as it relates to tax policy. 

With regard to the subject of this hearing today, in addition to 
trying to answer the 202-plus questions in the booklet that illus- 
trate the need to reexamine the base of the government, I do be- 
lieve there is strong conceptual merit to having some type of peri- 
odic assessment of programs, policies, functions, and activities. My 
personal view is that the agencies should be on the front line. This 
should be a normal and recurring part of their job. 

I think in order for these assessments to work, several things 
have to happen. It is important what the process is. It is important 
what the principles and the criteria are. It is also important who 
the players are that are involved in generating the results that will 
be considered by the Congress, as well as by the President, to the 
extent of the Executive Branch, in trying to make decisions. 

In that regard, I think there is a need for and an opportunity to 
institutionalize a periodic assessment process that builds upon the 
principles that are laid out in this booklet, including the generic 
questions, as well as the principles and generic questions that are 
outlined in the PART. By doing so we can try to come up with 
something that has a chance of being sustained over time and can 
generate more meaningful results, both financially and non-finan- 
cially, irrespective of what administration might currently be in 
power. 

Mr. Johnson. I think we have great opportunity to use the 
PART. Whatever it is called 5 years or 10 years from now there 
should be an instrument that we use consistently across all pro- 
grams to assess performance, and today is it called the PART and 
there are 25 questions, plus or minus. It should be a part of our 
job. It should be part of what agencies do on a day-in and day-out 
basis. That is the assumption in the PART now. 

Part of the PART program is you evaluate different aspects of 
the program and then you are supposed to develop recommended 
next steps, things that you, the program manager, are going to do 
this next year to improve performance, reduce cost, or both, and we 
are set up to monitor that and to hold agencies accountable for the 
follow-through — we, 0MB, are set up to follow through, hold agen- 
cies accountable and then follow through on those recommenda- 
tions. 

And some of these programs’ recommended next steps are really 
significant. They are going to invest more money in IP. They are 
going to reorganize this. They are going to change the rules, get a 



8 


bill changed, new legislation, or whatever. Others are less aggres- 
sive. 

They all should be held accountable for having a commitment to 
improve performance every year. That is built into the system. We 
are now at the point where we have some programs that are now 
in the second year and the third year since its asessment, so we 
are now able to start holding agencies accountable for that follow- 
up on what they said they could do to improve performance. 

The other thing we need to do and are preparing to do is to make 
this information more public so there is more discussion about 
PART and about what the desired outcomes are, what the perform- 
ance measures are, and so forth. All this information is available 
now on the website. You have to really want to find it and you 
have to be able to speak that form of English that only 0MB and 
a few Congressional staffers are adept at speaking. 

What we want to do is to take this information, just put it in 
English, put it in lay terms for all the world to see. Here is what 
we are spending your money on. Here is how we evaluate these 
programs. Here is how we measure success, and here is what the 
recommended next steps are. This more public information, we be- 
lieve, is going to drive more dialogue between the public and Mem- 
bers of Congress. 

There will be more dialogue between think tanks and good gov- 
ernment groups and Members of Congress and the Executive 
Branch about how much better this performance measure is than 
that performance measure, how much stronger and more aggres- 
sive this recommended next set of actions is than that one, and 
there will be more pressure, more dialogue, more discussion about 
how to drive performance even better. Make it more public, make 
it more transparent, and get more impetus behind using the infor- 
mation to drive performance. Both of those things — the first one ex- 
ists and the second one is in the process of being developed. 

Senator Coburn. It seems to me we have three problems. One 
is creating a culture where you use management tools of assess- 
ment and outcomes to drive policy, refinements, and efficiency 
within programs. 

The second is just the management expertise of demanding more 
with less, which is nowhere in any of this that I have found any- 
where. We still have yearly budgets that come in at baseline rather 
than zero-based budgeting. So that is the second part. 

And the third part is to engage Congress. The Congress is going 
to get surprised in about 4 years and we are going to be making 
major cuts to major programs to handle our financial difficulties. 
It seems to me whatever we can do to awaken Congress to what 
is about to happen to us in terms of international financial mar- 
kets, the force that is going to be placed on the Congress. 

My follow-up question is, what about the second and the third 
part? What about the more for less that is every business. Senator 
Lautenberg asked that of his business every year. Give me more 
for less. That is called efficiency. That is called productivity. He 
asked that and got it. What about that component of it? 

And the second question I would ask you about is how do you 
engage Congress? How do you engage Congress to address what 
those charts show, and how do we make the changes that are nec- 
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essary, create an awareness in Congress to do the reform? There 
are a lot of programs out there that are great. This isn’t going to 
be just about programs that are great ideas. It is going to end up 
being what about the programs that we cannot afford? Which is the 
best of those? That is the other reason why we should have such 
a good assessment tool. We cannot afford everything that we are 
promising today. 

So please answer those two subparts of that question, if you 
would, and then I will defer to Senator Lautenberg. 

Mr. Walker. Mr. Chairman, I would respectfully suggest that we 
need to integrate and institutionalize those two concepts into the 
current processes that we already have. For example, we have the 
Government Performance and Results Act, which requires strategic 
and annual performance planning, and annual performance and ac- 
countability reporting. A key component of that should be the con- 
cept of getting more done with whatever resources and authorities 
you have, and it could be with less, or it could be the same amount. 

Second, I think we also have to look at the budget process and 
the materials that are provided to the Congress. We need to move 
beyond baselines. The baselines are not sustainable. The baselines, 
to a great extent, represent an amalgamation and combination of 
programs, policies, functions, and activities that made sense when 
they were put into place but haven’t been subject to fundamental 
review and reexamination. Performance information needs to be in- 
cluded as part of that process. 

I think we need to make the agencies responsible for doing this. 
I think there needs to be a role for 0MB on behalf of the Executive 
Branch and the President, whoever the President might be. I also 
think there needs to be a role for GAO, because the fact of the mat- 
ter is that every administration has a President who is associated 
with a political party. They change from time to time, but by defi- 
nition, the Congress has to feel comfortable with the process, the 
principles, and the players. As a result it is important not just to 
get an Executive Branch assessment but also the GAO’s assess- 
ment since we are an independent, nonpartisan, nonideological 
agent and a subsidiary of the Congress. 

Mr. Johnson. When they first started using the PART 40 some 
odd percent of the programs could not demonstrate a result. So it 
was impossible to hold the head of programs accountable for the ac- 
complishment of a goal. They couldn’t define what the goal was. 
They didn’t know what they were trying to accomplish or they 
knew what they were trying to accomplish, but they couldn’t meas- 
ure it, or whatever, some combination of the above. 

With performance information for programs, you now have infor- 
mation that you can use to hold program managers responsible for 
the accomplishment of desired outcomes at desired costs, and it is 
at that point, and we are just now getting to that point, where you 
can start setting annual goals with program managers that we 
want to get more for less, or we want to get the same for less, or 
we want to get more for the same. And that needs to be part of 
holding managers accountable, holding employees accountable. 

That is one of the basic concepts for why there ought to be, we 
propose and will recommend here shortly, civil service moderniza- 
tion government-wide. It helps create an environment where peo- 
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pie, managers in particular, are held more accountable for how 
their programs perform, and also how to better engage Congress. 

When we started this, the whole PMA was designed to focus on 
opportunities to better manage the Federal Government that in- 
volved Congress initially as little as possible. Nothing against Con- 
gress, but we wanted to work as much as we could within the Exec- 
utive Branch to drive performance. Now there is more information 
available. There is information on 60 percent of the budget. We 
have information now that can be used to inform budget decisions, 
programs, continuation of programs, and elimination kind of deci- 
sions. So now there is enough information to more significantly in- 
volve Congress. 

So this is why we are having these kinds of hearings. This is why 
you are interested in this. We can get more buy-in about what the 
stated purpose is. We can get more buy-in to the validity of these 
performance measures. We can get more buy-in to the validity of 
these efficiency measures. 

And I think one of the key things that is going to lead to the 
credibility and validity of these numbers is if we are able to show 
Congress how this information is being used within the Executive 
Branch, not to eliminate, add, or subtract programs, but to drive 
performance. We can show that this makes sense. Program man- 
agers are using these new definitions of desired outcomes, and we 
are making these kinds of changes in the way programs are being 
managed and achieving different kinds of results, more desirable 
results, than we were several years ago before we had this informa- 
tion. 

I believe that is going to make a big difference in terms of how 
credible this information is with Congress, and then these kinds of 
trends also make it much more necessary for Members of Congress 
and the Executive Branch to pay attention to what is working and 
what is not, and what we are getting for the money. 

Senator Coburn. Senator Lautenberg. 

Senator Lautenberg. Thanks, Mr. Chairman. 

These two witnesses are very knowledgeable people. We see them 
with some degree of regularity. 

Mr. Johnson. We have the same agent. [Laughter.] 

Mr. Walker. We get the same fee. 

Mr. Johnson. The same fee. Senator, zero. [Laughter.] 

Senator Lautenberg. Well, we can discuss that at another time, 
but what I 

Mr. Walker. More for less. [Laughter.] 

Senator Lautenberg. I got a little concerned when Mr. Johnson 
made some reference, and I thought that there was a suggestion 
in there that maybe the Congress ought to be rated the same way, 
using PART. If that is the case, I will recuse myself from that hear- 
ing. [Laughter.] 

But in any event, thank heaven that we are not measured by the 
same yardstick. It is fair to say that there might even be some po- 
litical influence around here. 

How sure are you, Mr. Walker, can we be, that there is no polit- 
ical urging, to use the politest term I can, because this is all that 
I share with the Chairman here. We have gotten to know each 
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other some and we know that in terms of how we operate, there 
is consistency of view. 

As I listened to what each of you had to say, I thought about the 
differences in departments. How do you measure the museum per- 
formance and how do you measure a transportation program or the 
manufacturing program, which I think is a very good idea? You are 
left in kind of a discovery area. Who do you talk to — who gets spo- 
ken to when these decisions are being reviewed or questions are 
being asked? Who in the line of command typically gets talked 
with? Do the employees get to respond? Do the staff within these 
units get to respond, or is this strictly a management review? 

Mr. Johnson. I don’t know how far down it gets. It is both agen- 
cy and 0MB, but I don’t know how far down, if it gets down to the 
Forest Service person in whatever Western State. 

There are four basic areas of questioning in the PART that really 
are good questions that would apply to your form of business, my 
form of business, any kind of Federal program. Is there a clear, de- 
fensible purpose? There are several questions related to that. Are 
there valid short-term and long-term goals? Is the management 
sound or the management practices good? And what are the results 
of the program relative to the goals? Now, those are generic ques- 
tions that apply to any kind of program. 

There is work done by 0MB and people working on individual 
programs within the agencies. They get agreement or disagreement 
on what the ratings are. We are this year establishing an appeals 
board in case there is an impasse at what the evaluation ought to 
be, and it goes to a selection of deputy secretaries to review that 
and make some determination on what the ruling ought to be on 
that. 

But I feel comfortable that there are good assessment from both 
objective and pride of authorship. The people involved in the pro- 
gram standpoint, and the questions that are asked are generic in 
nature, but very focused on whether the programs are working or 
not. 

Senator Lautenberg. Mr. Walker. 

Mr. Walker. Senator Lautenberg, you having been in the private 
sector for many years and I having been in the private sector for 
21 years, these are management 101 concepts that apply to govern- 
ment as well as the private sector and the not-for-profit sector. 
They are not inherently partisan in nature. And yet the fact of the 
matter is, the government hasn’t done much in this area for many 
decades, irrespective of which party was in power. 

I would respectfully suggest that the agencies have to be pri- 
marily responsible and accountable for doing what needs to be done 
here. I would also agree that 0MB has a role to play. At the same 
point in time, I don’t believe that it can stop at 0MB. Given the 
fact that 0MB is part of the Executive Branch, and works directly 
for the President — whoever the President might be and whichever 
party that President might be associated with — I think you need to 
have a check and balance. I think part of that check and balance 
possibly is to have a role for GAO. Ultimately, it is not only going 
to require action by the Executive Branch, it is also going to re- 
quire action by the Legislative Branch in order to achieve meaning- 
ful and lasting results. 
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Rightly or wrongly, I know there is concern in the Congress with 
regard to just relying upon the PART. There is also interest in hav- 
ing some checks and balances in the process. That is how our Con- 
stitution is based. 

Senator Lautenberg. In the business world, again, each of our 
witnesses here has talked about the experiences in the private 
world. I think that the most reliable measure is to see how the cus- 
tomers like it. I don’t know whether we include that as part of our 
review. Again, I note that the museum is here, and I wonder, have 
we done any assessment or do we do any about visitors, whoever 
the customers may be. Unfortunately, it takes time to catch up 
with that opinion. The cart and horse thing that is so often used 
as a reference here is whether the resources are adequate to give 
the facility or the program enough time and enough direction to 
work effectively. 

Mr. Walker. Senator Lautenberg, I would respectfully suggest, 
as Clay Johnson touched on the criteria need to be customized to 
the particular entity involved. Let us take GAO as an example, 
which I know firsthand. We are a professional services organiza- 
tion. We have four primary measures. Measure one, results, finan- 
cial and non-financial results. Two, what do our clients, meaning 
the Congress, think about our work? Three, what do our employees 
think about our agency? And then four, what do our partners, both 
within government and outside of government domestically and 
internationally, think about us? 

Those four measures work very well, and there are a lot of de- 
tails behind those measures, but the framework has a lot of concep- 
tual merit. However, the details obviously have to be tailored to the 
particular enterprise and their workforce. 

Senator Lautenberg. And each of you is satisfied that there is — 
that these results are free of any skewing for political or ideological 
decisions? 

Mr. Johnson. I am totally satisfied that they are free of those 
biases. 

Mr. Walker. We haven’t evaluated that. I believe that to the ex- 
tent that you have a process that provides for checks and balances, 
then it can provide additional assurance, not just for today, but 5 
years from now, and 50 years from now. 

Mr. Johnson. I want to talk about customer service in a second, 
but one thing, there is less bias in our assessment of the ratings 
than will be there in Congress’ assessment of whether programs 
work or not, to your point that you made earlier. 

On customer service, we had talked about that. I can tell you 
that every program that has a large customer service component, 
which is almost all of them, does a lot of customer service meas- 
uring. All of our lending operations, all of our grant-making oper- 
ations, all of our e-Government initiatives are measuring customer 
satisfaction, and like many government issues, they are comparing 
our ability to take a reservation for a campsite with orbit.com’s sat- 
isfaction with their ability to take an airline reservation. So we are 
looking for private sector benchmarks and trying to build that into 
program managers’ goals. 

The thing we have to realize, our ability to assess programs are 
in a pretty infant stage. We are now in a 3-plus year process of 
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measuring program performance. We haven’t even evaluated all 
the programs yet. The program metrics and measurements we use 
will be way better 5 years from now than they are now and better 
still 10 years from now. We will get better at measuring. We will 
get better at correlating this with results and so forth. 

We talked about how we don’t have customer satisfaction meas- 
ures in all of our customer service programs. It is in some, but not 
all. That needs to be corrected. That will be corrected. But we are 
going from a situation 2 years ago where 50 percent of the pro- 
grams could not demonstrate any result, forget the right result, 
any result. We are now coming to the point where we have meas- 
ures, a lot of them are very rudimentary and basic, and now we 
are going to build from there and make them much better and use 
that information to hold the program managers more accountable 
for the delivery of more for less. 

Senator Coburn. Senator Carper, welcome. 

OPENING STATEMENT OF SENATOR CARPER 

Senator Carper. Thanks, Mr. Chairman. I apologize for running 
a little late. Our caucus luncheon ran until about 2:15. We had a 
heated discussion on energy policy, and as I was making my way 
over here, my cell phone went off and it was our Lieutenant Gov- 
ernor from Dover, Delaware calling, from Legislative Hall, to give 
me the results of the elections in Boys’ State and he was standing 
next to the newly-elected Boys’ State Lieutenant Governor, our old- 
est son, Christopher Carper, who we are just very proud of 

Senator Coburn. All right. 

[Applause.] 

Senator Carper. Nobody has asked for a recount. [Laughter.] 

So, I think we are in a moment of real pride. He had lunch at 
the Governor’s house today, where we used to live. Tuesday in 
Delaware, we have a legislative leadership luncheon where Demo- 
crat and Republican leaders in the legislature and the Governor 
and the Lieutenant Governor all have lunch, and sitting right there 
at the table was — literally in a room where he grew up for 8 years, 
was our boy, real special. 

I have a statement I would like to give, not now but maybe dur- 
ing our break between our panels, if you don’t mind. I want to say 
to our witnesses that I don’t think I have ever seen the two of you 
before, but it is great to see you. [Laughter.] 

People are going to start talking, b^ut we are glad you are both 
here. Thank you for your testimony and your response to our ques- 
tions. 

Let me just start off, if I could, with a question for you, Mr. 
Johnson, and I am going to ask Mr. Walker to respond to it, as 
well. I will give you the first shot at it, if I can. 

I have read criticisms, and you have probably heard them, too, 
of the Program Assessment Rating Tool which argue that OMB’s 
ratings sometimes ignore the will of Congress with respect to our 
intent in creating certain programs. I just wonder, is it possible for 
a program to get a poor rating simply because it does what is re- 
quired by statute and not necessarily what 0MB might like for 
that program to do? 

Mr. Johnson. Yes. 
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Senator Carper. All right. What should we do about that? 

Mr. Johnson. What we challenge our agencies to do is 

Senator Carper. I want the record to show, that is the shortest 
answer I have ever gotten from any witness in the 4-plus years I 
have been here. Could you be more direct? [Laughter.] 

Mr. Johnson. We charge agencies to have outcome goals for each 
program, and if the enabling legislation doesn’t provide that or 
leaves that vague or there are contradictory outcomes called for, we 
make it the responsibility of the agencies to go back to the appro- 
priate body here and fix that. 

They say, well, but I can’t control that. Then we say, well, we 
are not saying who is at fault here. We are saying we have a pro- 
gram that does not have a defined outcome, or it has a defined out- 
come and perhaps in some cases the measures are undefined — or 
the goals are so broad or so vague that you could never hold any- 
body accountable for their accomplishment. That needs to be fixed. 
And if it involves working with Congress to fix it, then it is their 
responsibility to work with Congress to fix it. 

Again, the assessment is not of Congress — it is of the program, 
but the program manager is responsible for doing what they can 
to get the most result for the amount of money spent for that pro- 
gram, and if it involves corrective legislation, that is what they 
need to go try to get. 

Senator Carper. Mr. Walker, any thoughts, please? 

Mr. Walker. First, I think one of the things that Congress needs 
to do when it is enacting legislation, whether it be a new program, 
or whether it be a new tax policy, is it needs to think about why 
is it doing it, what is it trying to achieve, and how should success 
be measured. Therefore, at the very creation of a program or policy, 
these are issues that should be focused on by the Congress, since 
it is the body that causes it to be created and appropriates money 
on a recurring basis. That is generally not being done at the 
present time. 

Second, it is important that we not just look at the programs 
with regard to the different departments and agencies, but we need 
to have additional emphasis on horizontal activities, because many 
times there are many programs that are being operated and trying 
to accomplish a similar goal within many different departments 
and agencies, and we need to focus on employing more of a hori- 
zontal and integrated approach. 

Last, I think we can’t forget about tax policy. We spend as much 
in this country, or fore go as much in revenue in this country in 
some years as a result of tax expenditures and tax preferences as 
we do in total discretionary spending. Discretionary spending in- 
cludes national defense, homeland security, judicial system, edu- 
cation, the environment, GAO, 0MB, etc. It is important that we 
not let tax preferences/expenditures off the radar screen. They have 
to be on the radar screen, too. 

Senator Carper. I have to ask this question. It is not really ger- 
mane to our hearing today, but you mentioned tax expenditures 
and revenue flow gone. Is there any significant revenue flow gone 
simply because taxes that are owed and are not being collected? 

Mr. Walker. Over $300 billion is the estimated tax gap, of which 
there are sub-elements to that, some because people aren’t claiming 
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the income, some because they have understated their gains, some 
because they have overstated their losses, some because they have 
delinquencies that we haven’t collected. And we had a hearing on 
that before the Senate Finance Committee and that is an issue 
that needs more attention, as well. 

Senator Carper. So $300 billion, is that like a one-time number 
or is that a recurring number? 

Mr. Walker. Every year. 

Senator Coburn. The Chairman and I have talked about that 
and I think that might be the subject of some subsequent inquiries 
not today, but that is an issue for another day. 

We had another hearing in our full Committee, Mr. Chairman, 
this morning, and out of the mouths of a couple of our witnesses 
came the words “Government Performance and Results Act,” and 
I don’t remember anybody ever mentioning that in a hearing be- 
fore, at least not that I can recall. Anyway, it was mentioned a 
time or two in our earlier hearing. 

What I would like to do is ask, if I could, just start with Mr. 
Johnson to ask a little bit about how we coordinate the rating tool 
with the Government Performance and Results Act, which I think 
might have been adopted about a dozen or so years ago but I think 
it goes back to the early or mid-1990s. How does 0MB take into 
account programs’ successes in fulfilling obligations, placed on 
them in their agency’s long-range or annual performance plan? 
Does the rating tool ignore the Government Performance and Re- 
sults Act? How do we just coordinate the two and use them both 
effectively, or is that impossible? 

Mr. Johnson. No, it is not impossible at all. In fact, they should 
be used together. The Government Performance and Results Act 
was adopted 12 years ago and 

Senator Carper. Could you just take a second and give us a little 
short primer on the Government Performance and Results Act, if 
you are able to? And if you are not, you are OK. 

Mr. Johnson. I am known to be a pretty good delegator, and so 
I am going to call on my good friend, Mr. Walker. 

Mr. Walker. That is called delegating up. Senator, but that is 
OK. [Laughter.] 

Mr. Johnson. Whatever you want to call it, just do it for me. 

Mr. Walker. He is very adept at delegating. 

Senator Carper. Well, we have all done it. 

Mr. Walker. I would say. Senator, one of the things that I men- 
tioned before was I think it is very important that we recognize 
that we do have a number of statutory provisions in place, one of 
which is the Government Performance and Results Act, GPRA. My 
personal view is that we need to take a lot of these concepts, and 
integrate them into the current framework. We also need to insti- 
tutionalize them such that they will exist irrespective of what ad- 
ministration is in power and irrespective of which party controls 
the Senate and the House of Representatives. 

I do not think they are mutually exclusive. They should both be 
done. One should be a subset of the other. 

Senator Carper. How did he do? 

Mr. Johnson. It wasn’t much of a summary, but [Laughter.] 

Mr. Walker. Oh, you want me to summarize GPRA? I apologize. 
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Senator Carper. Just a primer, if you would, on 

Mr. Walker. Yes. The Government Performance and Results 
Act, which I believe was created in 

Mr. Johnson. I would give him a 

Senator Carper. An incomplete? 

Mr. Johnson. No 

Senator Carper. You are taking my time. 

Mr. Walker. The answer is that the Government Performance 
and Results Act, which I believe was created in 1993, provides for 
a number of things. One, it provides that agencies do a strategic 
plan periodically. Second, it provides that agencies prepare an an- 
nual performance plan. Third, it requires that agencies publish an 
annual performance and accountability report. 

I might note for the record that, as frequently is the case, many 
times, these pieces of legislation are passed and only apply to the 
Executive Branch. In this particular case, as in many others, we 
have voluntarily adopted it at GAO, not just to comply, but to try 
to lead by example. 

It was a very valuable concept and we have made a lot of 
progress in the last 12 years with regard to this. But we still have 
a ways to go. I believe that some of the concepts we are talking 
about today represent a prime example of how we need to integrate 
these concepts into our existing mechanism. However, we need 
greater involvement by the Legislative Branch, including the Con- 
gressional committees, because to a great extent, whether it is 
budget, whether it is appropriations, whether it is authorizations 
or reauthorizations or whether it is oversight, this type of informa- 
tion is not used to the extent that it should be. 

Mr. Johnson. David and I were on a panel a couple of years ago 
with Pat McGinnis with the Council on Excellence in Government 
and Congressman Armey talking about the Government Perform- 
ance and Results Act, and I think that it was the feeling of every- 
body on that panel that the Government Performance and Results 
Act had not lived up to its potential. 

I think one of the reasons is that the unit of evaluation, I sug- 
gest, should not be an agency. It is difficult to say that an agency 
has succeeded, an agency is the sum of its programs. There are 
some agencies like Commerce that have the most unbelievably 
wide diversity of programs — they all do, but some of them are in- 
credible. And to say that Commerce is doing this or doing that 
overall, it is not really relevant. 

So I think now having program information gives us information 
about relevant units of measure, relevant component parts of agen- 
cies that can now be incorporated into our overall discussion about 
if Commerce, if Interior, if the State Department is meeting its 
mission? Is it accomplishing its strategic goals? 

Three years ago, we only had program information for 20 percent 
of the programs, then 40, now 60. We are in 80. This next year is 
for us the time when all of us should figure out how to really bring 
these two things together, because I think it gives us an oppor- 
tunity to realize even more completely the full potential of the Gov- 
ernment Performance and Results Act of 12 years ago. 

Senator Carper. Yes, sir. 
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Mr. Walker. One last thing, Senator Carper, we still need a gov- 
ernment-wide plan. We don’t have a government-wide strategic 
plan. We don’t have a government-wide performance plan. We do 
have, I would argue, a performance and accountability report, but 
we still need the government-wide plan. 

The budget is, by definition, not a strategic plan because it 
doesn’t look out far enough. 

Senator Carper. Who should be responsible for developing that? 

Mr. Walker. The President, and presumably the President 
would use his very valuable agent, 0MB. 

Senator Carper. Thank you. Gentlemen, thanks very much. 

Senator Coburn. I want to go back for just a moment. Mr. John- 
son, in your testimony, you stated that the Administration is pro- 
posing two new commissions, the Sunset and the Results Commis- 
sions. Can you talk in some detail about what those commissions 
would do, how would they be created, and how they would be struc- 
tured, and what their purposes would be? 

Mr. Johnson. Yes, and I know I am going to miss a few of these 
facts. The Sunset Commission is a concept, I think it is employed 
by about half the States. Texas is one. What we are proposing is 
that there be a seven-member commission, four in the majority 
party — or in the Administration, three not. Four members ap- 
pointed by the President with consultation from the majority and 
minority leaders in both Houses. 

There would be a list put together with Congress, a schedule of 
when programs or agencies would come up for review over a 10- 
year period of time. So it would set a 10-year review cycle. If Con- 
gress and the President agreed that the department of X ought to 
be reviewed in total, it will be on that list. If they agree that it only 
ought to be programs or it ought to be these programs but not 
those programs, or it shouldn’t be defense programs, or it should 
be — that will be worked out, but there will be a 10-year review 
cycle. 

Then every year, those programs, and so we are talking about 
1,200 programs — let us say it is only program specific — it would be 
1,200 programs. About 120 programs a year would come up for con- 
sideration. If they are not affirmatively agreed to to be continued, 
they would sunset. 

My understanding is that the experience in the States is that the 
programs tend not to then go away. They tend to know that their 
judgment day is coming up, so there is accountability — Congress is 
on the line, calling for performance. The Executive Branch is on 
the line, calling for performance. Program managers really feel 
held accountable for how their programs perform, and it tends to 
drive more significant performance, more focus on results, and 
more focus on efficiency. 

And so what happens is programs get better. Improvements get 
better. Occasionally, the Sunset Commission will come out with a 
recommendation to modify a program or change the measures or 
less of this and more of that as opposed to this thing should go 
away altogether. If something needs to be done away with, my 
guess is, and I think the experience in States is, it will generally 
be done away with long before it ever comes up before sunset re- 
view. So it is a mechanism for driving a formal focus on results. 
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Congress would be involved and the Executive Branch involved, 
both calling for results. 

The Results Commission is to deal with the programs like job 
training or community and economic development or programs 
dealing with disadvantaged youth or world water quality, where 
there are issues that many agencies and many programs are in- 
volved with which makes it very difficult for the Executive Branch 
and for the Congress to deal with these matters, because you get 
into jurisdictional issues, you get into so many different budgets, 
you get into issues about, well, is it Commerce’s fault, is it Inte- 
rior’s fault, is it whatever. 

If Congress agreed to the concept, then what would happen is the 
President would propose that we ought to create a specific Results 
Commission on the subject of job training. Congress would agree 
that was not too controversial a topic or it was a good enough 
amount of money or a substantive issue that we should create a 
Results Commission to deal with that. They would agree to do that. 
The President would form a seven-person commission, four in con- 
sultation with the majority and minority leaders in both Houses. 
But it would be seven people who have expertise of various sorts 
on the subject of job training in this case. It would be a commission 
put together to deal with the specific issue at hand. 

They would then have 9 months to receive a proposal on how to 
organize these multi-agency, multi-program efforts differently, do 
away with some, add some things, combine them over here, com- 
bine them into — whatever the recommendation is, take that, have 
hearings, decide what they believe — how the President’s rec- 
ommendation ought to be amended, if at all, come back to the 
President with that. There is some dialogue back and forth. Any- 
way, but then that goes to the Congress for expedited consider- 
ation, like a BRAC-kind of a process. 

So one deals with single agency kinds of performance issues. The 
other one deals with single program performance. The other one 
deals with multiple departments, multiple agencies, multiple pro- 
grams dealing with the same issue. 

I have left a lot of the details out, but 

Senator Coburn. Talk with me for a minute about the fact that 
we are going to approve $170 billion worth of spending this year 
for programs that aren’t authorized. How does the Results or the 
Sunset Commission deal with that? In other words, we are going 
to spend $170 billion. We are going to appropriate it this year for 
programs that are totally unauthorized. In other words, the Con- 
gress hasn’t done this job, either won’t reauthorize them, don’t 
have the votes to reauthorize and appropriate them anyway. How 
do you address that problem? Thirty percent of our discretionary 
budget is appropriated without any authorization. 

Mr. Johnson. I am not familiar with the particulars, but when 
the Congress and the Executive Branch are developing that list of 
programs, that schedule to review all programs, it would decide 
whether they are reviewing just authorized programs, or unauthor- 
ized programs. They might decide to put all those unauthorized 
programs in the first couple years of sunset review. 
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So Congress is integrally involved with the Executive Branch to 
decide what is involved and in what order these different programs 
are involved. So this would be a mechanism for doing that. 

Senator Coburn. We really could do that if we had effective over- 
sight, without either of those commissions, couldn’t we? 

Mr. Johnson. Yes, sir. 

Mr. Walker. Mr. Chairman, I would respectfully suggest the 
commission wouldn’t deal with that issue. The fact of the matter 
is that you already have a number of programs that have not been 
reauthorized yet, even though they are supposed to be reauthor- 
ized, and yet the Congress has decided to continue to appropriate. 

There is absolutely no question that we need additional emphasis 
on whether or not programs, policies, functions, and activities are 
achieving real results. There is absolutely no question that we need 
additional emphasis on mechanisms to determine whether and to 
what extent programs should be continued. 

However, we should integrate those mechanisms into other ongo- 
ing processes and institutionalize them. I haven’t seen the proposed 
legislation and I would reserve any comment on the proposed legis- 
lation until I see it. However, historically, we have done a lot of 
work at GAO with regard to commissions that work and commis- 
sions that don’t work. The commissions that typically have worked 
the best are the ones that had a finite term and a specific mission 
to be accomplished. What I am hearing is something that could go 
on indefinitely. 

The real key is, how can we end up making sure that the Con- 
gress is engaged to a greater extent? How can we make sure that 
more of this type of information is automatically considered? How 
can we increase the transparency of this information, not only for 
the benefit of the Congress, but also for the benefit of the public 
so there are incentives to start dealing with some of the tough 
issues that have accumulated over time and that we are going to 
be forced to deal with because of the fiscal picture that I outlined 
earlier? 

Senator Coburn. Senator Carper. 

Senator Carper. Just a couple more questions, if I could. Let me 
just ask, and again, I sort of direct this to you, if I could, Mr. John- 
son, and then to ask Mr. Walker to respond, as well, but what fac- 
tors do the views of stakeholders or the individuals or groups that 
benefit from a given program, what factors would their views play 
with respect to OMB’s ratings of a particular program and how, if 
at all, does 0MB actually solicit their views? 

Mr. Johnson. I know that we pay attention to if programs are 
working or not, and if they are supposed to serve a given commu- 
nity, the most important stakeholder group is the community of 
citizens that are supposed to be served, and so there are different 
measurements of whether they are being served or not. 

But the exact method of doing that and the degree to which it 
happens across the board, I don’t know, but I can get back to you 
with that information. 

Senator Carper. General, do you want to take a shot at that one, 
as well? 

Mr. Walker. He would be in a better position to say on PART. 
I do believe that it is a factor that should be considered. At the 
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same point in time, just because you have a lot of people that are 
happy that the program exists doesn’t mean that the program or 
policy, if it is on the tax side, is achieving the desired results. 

All too frequently, in the absence of having performance and out- 
come-based information, the assumption is if we spend more 
money, we will get more results, or if we give more tax preferences, 
we will get more results. That is not necessarily true, and that is 
part of the whole purpose here. We need to find out what works 
and what doesn’t work because we are not going to be able to af- 
ford and sustain all that we have right now. We also need to make 
room for other things that the Congress will want to do because of 
emerging needs and challenges facing the country in the future. 

Senator Carper. OK. And one last question, if I could. Based on 
the analysis of the rating tool scores, grant programs, as I under- 
stand it, are less likely to receive effective scores and much more 
likely to be deemed ineffective or to receive a score of results not 
demonstrated. I just wonder, why do you think this is, and how do 
rating tool questionnaires differentiate between different types of 
programs? 

Mr. Johnson. There are questions that are asked of grant pro- 
grams that are specific to grant programs. I was giving a talk to 
the SESes at EPA about a month ago and a couple fellows came 
up to me at the end and said PMA is great and getting more re- 
sults and better defined goals and so forth. We need to get States 
and local communities to, once we grant them the money, get fo- 
cused on getting more for the money. 

And I said, what is fabulous is that you are asking me about 
this. I bet you 5 years ago, you never would have been inclined to 
think about how we can now focus on what our grant recipients do 
with the money, because that is what we are holding agencies ac- 
countable for. The goal is not to give monies to States and munici- 
palities in an efficient, effective fashion. The goal is to get good 
things happening with the money that States and municipalities 
are spending in a prescribed nature. 

So that means not only we have to define what the goals are, but 
we have to hold States and municipalities, to the extent to which 
we can, accountable for spending the money as designed and in an 
effective fashion. Right now, we are not particularly equipped to do 
that, and I think that disconnect between us giving them the 
money and them spending the money wisely or them spending the 
money as intended or them producing or measuring the results of 
that is not what it needs to be. But now our agencies know that 
they are being held accountable for how their grant recipients 
spend the money and so they are going to turn around and work 
with States or municipalities and maybe make the money they get 
next year dependent on how well they spend the money this year. 

We can get better at this. We just generally have not been. That 
has not been our goal, is to seek the performance against desired 
outcomes at the bottom level. 

Senator Carper. All right. Mr. Walker. 

Mr. Walker. Several thoughts. Senator. One, there are different 
forms of grants, including flock grants, and I think one of the 
things that we need to keep in mind is that we need more perform- 
ance-related information, we need more transparency so that we 
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can have more accountability both at the Federal level as well as 
the State and local level, depending upon the facts and cir- 
cumstances. 

Second, we have a finite amount of resources, which are going to 
get tighter as time goes forward based upon known demographic 
trends, rising health care costs, and other factors. As a result, we 
need to be more value and risk oriented. We also need to be more 
targeted with regard to expenditures as well as tax preferences. 

The last thing is, and it is a concern that I have, there are other 
practices that exist that can complicate this. For example, the prac- 
tice of Congressional earmarking. When you are in a situation 
where you have a finite amount of resources and it is going to get 
tighter and you are trying to get people to focus on results, to the 
extent that there is going to be more earmarking, then there is 
going to be less flexibility to be able to target, to achieve desired 
outcomes, to mitigate risk, which could establish a vicious cycle 
that should be of concern to all parties. 

Senator Carper. Good enough. Thank you both very much. 

Senator Coburn. Thank you. I am going to submit some ques- 
tions in writing so we don’t carry this out too long. 

The other point I would make, the number one stakeholder in all 
these programs is the next two generations. It is not the stake- 
holders that are meant to serve, because if we don’t solve the finan- 
cial problem, there is not going to be any service and the stake- 
holders, the next two generations, are going to be paying the bill 
for things that we have already spent the money long ago. 

So it is important for us to — the ultimate stakeholder is the 
American taxpayer for all these programs, because since we are 
running on a deficit to the tune of about $22,000 per man, woman, 
and child per year in this country, and the people sitting in this 
room, the vast majority, aren’t going to be paying that money back, 
it is going to be our children or grandchildren, it is important to 
keep perspective of who the real stakeholder is. 

I want to thank you for your testimony. I am sure we will invite 
you back. We appreciate so much you being with us, and with that, 
we will dismiss the first panel. 

Senator Carper, while our second panel is coming up, I will ask 
you to 

Senator Carper. I think I would just ask unanimous consent 
that my statement appear in the record and we will go right to the 
second panel. 

Senator Coburn. Without objection. 

Senator Carper. Thanks. 

[The prepared statement of Senator Carper follows:] 

OPENING prepared STATEMENT OF SENATOR CARPER 

Thank you, Mr. Chairman, for holding this very timely hearing. 

As you and all of our witnesses are well aware, our country is facing record budg- 
et deficits. We’re just embarking on another appropriations season here in Congress 
where we’ll be called on to make some very difficult decisions about what to do with 
scarce Federal dollars. 

At the same time, as GAO and General Walker have pointed out countless times 
before this Subcommittee and elsewhere, we’re at a kind of turning point right now 
where we need to decide what we want our government to do in the 21st Century. 
Nearly 4 years after the attacks on September 11, we still have a whole new set 
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of needs and priorities that must be balanced against older needs and priorities and 
scores of popular programs. 

And with the challenge of the retiring Baby Boom generation on the horizon, we 
just can’t afford to do all of the things we might want to do — at least not well. 
That’s why proposals like OMB’s Program Assessment Rating Tool are so inter- 
esting. 

We should never be afraid of taking a hard look at Federal programs to determine 
whether or not they’re accomplishing what we intended for them to accomplish 
when we first created them. In this day and age, we simply can’t afford to allow 
poorly-managed programs to continue without reform or, frankly, for a program that 
has run its course and achieved its goals to continue draining resources from other 
priorities. That said, we need to be certain that the Program Assessment Rating 
Tool or whatever mechanism we use to make these evaluations is itself effective. 

To be effective, a program like the Program Assessment Rating Tool must be to- 
tally separated from politics and ideology. It must also be closely coordinated with 
existing mechanisms agencies and Congress use to align budgets with program 
goals and outcomes, such as the Government Performance and Results Act. And per- 
haps most importantly, we also need to make sure that a program’s intended bene- 
ficiaries outside of Washington have a say before an evaluation is completed. While 
I’m keeping an open mind on this issue, I’ve some evidence that the Rating Tool 
might fail all three of these tests. 

I’d just say in closing, Mr. Chairman, that we’re not going to close the budget def- 
icit by reducing spending on a program here or eliminating a program there. Presi- 
dent Bush called for the curtailment or elimination of 154 programs in his Fiscal 
Year 2006 budget proposal. Even if Congress were to eliminate every single one of 
those programs, I think the savings would only cover a fraction of our budget deficit. 

Non-defense discretionary spending, the target of many of the spending reductions 
and program eliminations in the President’s Budget proposal, makes up only about 
16 percent of the Federal budget. I’m sure we can find ways to improve the manage- 
ment of some of the funding in that 16 percent, or even to find and eliminate waste 
or inefficient use of resources. If we truly want to tackle the fiscal problems facing 
us right now, however, we — meaning Congress — need to take a look at the entire 
budgetary picture, on both the spending and revenue side, and make some tough 
decisions. 

Senator Coburn. Our second panel, we are privileged to have 
with us today two scholars from the academic community to give 
us their perspectives on Federal Government accountability efforts. 

Our first witness on the second panel is Eileen Norcross from the 
Mercatus Center in George Mason University. Much of Ms. 
Norcross’s academic research effort has focused on the analysis of 
budget and performance integration and agency performance re- 
ports. We look forward to hearing from her today. 

Our second witness on the panel is Dr. Beryl Radin, Professor of 
Government and Public Administration with the University of Bal- 
timore. She has written extensively on the role of the Federal man- 
agement as an instrument of policy implementation, and we look 
forward to hearing her thoughts on the current initiative to make 
government more accountable and outcome-oriented. 

Ms. Norcross, if you would. 

TESTIMONY OF EILEEN NORCROSSP RESEARCH FELLOW, 

GOVERNMENT ACCOUNTABILITY PROJECT, THE MERCATUS 

CENTER OF THE GEORGE MASON UNIVERSITY 

Ms. Norcross. Thank you. Chairman Coburn and Senator Car- 
per and Members of the Committee on Homeland Security and 
Governmental Affairs, for inviting me to testify on the state of ac- 
countability and results in Federal budgeting. 

Much of my research at the Mercatus Center is on the progress 
agencies have made towards developing outcome measures, the 


The prepared statement of Ms. Norcross appears in the Appendix on page 60. 
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Program Assessment Rating Tool, and the extent to which Con- 
gress and the Executive Branch use performance information in 
the budget. I would like to submit for the record my paper ana- 
lyzing the results of the fiscal year 2006 PART. Additionally, I 
would like to submit a copy of our 2004 annual scorecard for your 
reference. 

Why should we bother to evaluate the performance of agencies 
or program results? What is the purpose of linking performance in- 
formation with cost information? Policy makers can and do debate 
values and priorities, but such a debate is about ends, not means. 
Policy makers often articulate many worthwhile ends — reducing 
homelessness, eradicating disease, or ensuring we are secure from 
terrorism. If we are to accomplish these ends, we must know if re- 
sults are being achieved. This means holding programs to a fact- 
based, not values-based, standard. 

If our goal is to reduce homelessness, does the program actually 
do this? If so, to what extent does it succeed? Do other approaches 
reduce homelessness more effectively? How many fewer homeless 
people would we have if we moved resources from a less-effective 
to a more-effective program? 

Answering such questions requires reliable performance and fi- 
nancial information. Linkage of results information and cost infor- 
mation tells us which means are most likely to accomplish the ends 
policy makers decide are worthwhile. Measuring results enables us 
to know what public benefits arose from a given activity. 

Agencies are statutorily required to report on their annual per- 
formance by articulating their goals, designing performance meas- 
ures, and assessing results achieved. This then requires that agen- 
cies link their goals, objectives, and performance measures with 
their budgets. 

According to our annual scorecard, which evaluates how well 
agencies are meeting their reporting requirements, we find many 
agencies lag in linking performance information and financial infor- 
mation. Not until our third annual scorecard in 2002 did an agen- 
cy, SBA, receive the highest score in this category. They achieved 
this because each of their performance indicators included a cost 
estimate, and some included a cost per output measure. This prac- 
tice continued in 2003. 

This measure is important because where there are programs of 
equal efficacy, then the best means of comparing them is the cost 
per unit of success. This allows us to know how resources might be 
used to increase the public benefit. 

Some agencies have made improvements in recent years. In our 
first scorecard in 1999, 14 agencies showed no linkage of cost to 
goals. This year, 7 agencies allocated their costs among goals and 
objectives, falling short of the highest score only because they failed 
to link these costs to individual performance measures. 

With the fiscal year 2004 budget, the Bush Administration at- 
tempted to forward the use of performance budgeting with PART. 
By formally linking budget requests to program performance, 
PART provides a view into how the Executive Branch is making 
some of its budgetary decisions. 

This approach has several merits. It is program-focused. Budget 
decisions are often made at the program level. Where multiple pro- 
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grams attempt to accomplish similar outcomes, PART assessments 
can facilitate comparisons. 0MB has made the assessments pub- 
licly available, ensuring the process is transparent and open to the 
public. 

There are some shortcomings. We may agree or disagree on indi- 
vidual PART assessments. The yes/no format may oversimplify 
agency answers. There are difficulties in relating individual pro- 
gram assessments to GPRA’s assessment of performance goals. 

PART must remain open to constructive criticism in terms of its 
methodology and mechanics, but it would be a setback to the use 
of performance information if the concept of trying to objectively as- 
sess program results were abandoned altogether. 

In the fiscal year 2006 budget, the President recommended 155 
programs for termination or program cuts. Fifty-four of these pro- 
grams have been PARTed. Of the 99 recommended for termination, 
32 have been PARTed. Of the 55 recommended for cuts, 23 have 
been PARTed. These 55 programs represent about $10 million in 
savings, or 0.4 percent of the proposed $2.57 trillion budget. Within 
the 55 programs, it appears PART was used in conjunction with 
other information to make funding decisions. Twenty-five of the 
175 programs rated results not demonstrated, and half of the 22 
programs rated ineffective to date were recommended for elimi- 
nation or cuts. There is not a perfect correlation between a PART 
score and funding decisions. 

PART is not the only means to better integrate performance in- 
formation into the budget. Important criticisms remain about its 
methodology, question format, and ratings classifications. But 
PART remains a consistent, systematic, and transparent attempt to 
evaluate government programs. By focusing on individual programs 
where budgetary decisions are often made, PART is a valuable ap- 
proach that can only improve the effort to advance performance 
budgeting. 

Thank you, Mr. Chairman, Senator Carper, and Members of the 
Committee for taking on this important subject. The integration of 
performance information into the budget is a vital means of guar- 
anteeing that the stewardship of public funds will achieve the most 
effective results and show the greatest public benefits. I hope this 
testimony will be helpful as the Subcommittee considers the role of 
performance information in the Federal budgetary process. 

Senator Coburn. Dr. Radin. 

TESTIMONY OF BERYL A. RADIN, i PROFESSOR OF GOVERN- 
MENT AND PUBLIC ADMINISTRATION, UNIVERSITY OF BAL- 
TIMORE 

Ms. Radin. Chairman Coburn, Ranking Member Carper, my 
name is Beryl A. Radin and I am a Professor of Covernment and 
Public Administration at the University of Baltimore and an elect- 
ed Fellow of the National Academy of Public Administration, and 
I will be joining the faculty at American University’s School of Pub- 
lic Affairs this coming fall. 

For more than a decade, I have been studying the efforts within 
the Federal Government to improve the effectiveness and account- 


^The prepared statement of Ms. Radin appears in the Appendix on page 74. 
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ability of Federal programs and have published a number of arti- 
cles on this subject. Like many others, I believe it is important to 
find ways to assure that limited Federal dollars are used effectively 
to carry out the goals and objectives of programs that have been 
created by both the Congress and the Executive Branch. But while 
the focus on performance is extremely important, I have serious 
questions about the current procedures that have been put in place 
to carry it out. 

The effort that has been undertaken in 0MB through PART may 
have been motivated by a le^timate and appealing concern, but I 
do not think that this model is an appropriate way, or is a limited 
way to measure program performance. 

The six issues that I will discuss today explain why there ap- 
pears to be a disconnect between many of the PART evaluations 
overseen by 0MB and the budget proposals that were found in the 
President’s current budget. They illustrate how difficult it is to im- 
pose a single model in an extremely complex Federal system with 
a diverse array of programs. We should heed what H.L. Mencken 
once said, “Explanations exist. They have existed for all times, for 
there is always an easy solution to every human program, neat, 
plausible, and wrong.” 

Let me summarize my six points. First, many Federal programs 
have multiple and conflicting goals. The PART process does not 
really reflect that reality, and most of the evaluations that have 
been done assume there is a single goal for programs. 

Second, not all Federal programs are alike. There are major dif- 
ferences, and we heard some of that earlier, between competitive 
grant programs, block grant programs, research efforts, regulatory 
programs, and other program forms. Yet the PART approach large- 
ly treats them alike, even though 0MB acknowledges the dif- 
ferences and GAO has actually written about that extensively. Per- 
haps most importantly, the PART process does not recognize the 
decisions by Congress to enact programs in different form. Instead, 
0MB actually second-guesses Congress in terms of assessment of 
program purpose and design. 

My third point, 0MB budget examiners and 0MB itself have a 
limited perspective on programs. It does not make sense to rely on 
only one perspective to determine whether programs should live or 
die. Congress itself has recognized that as it has separated the au- 
thorizing and appropriating functions. A yearly budget is not the 
only way to look at what are often very detailed and complex pro- 
grams. 

Fourth, there are many different types of information that are 
useful to those who are charged with running or assessing pro- 
grams. The information that is used in the PART process is not 
value-neutral. Rather, it reflects markedly different reasons for 
concern about performance. Significantly, the information that is 
emphasized by 0MB often is not useful to program managers, pol- 
icy planners, or evaluators, or judging from the quite tepid reaction 
on Capitol Hill, to those charged with appropriations recommenda- 
tions. 

Fifth, 0MB calls for new data sources but does not acknowledge 
that agencies are not able to collect that data. A number of agen- 
cies would like to collect data on the achievement of program out- 
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comes. However, they are constrained both by the mandates of the 
Paperwork Reduction Act as well as by their inability to receive ap- 
propriations that would give them the sources to develop these 
data systems. 

And sixth, PART focuses on an Executive Branch perspective and 
is not easily transferred to the Congressional branch. The one-size- 
fits-all approach that is found in the PART process is not compat- 
ible with a Legislative Branch with multiple committees and sub- 
committees as well as separating between authorizing and appro- 
priations perspectives. The multiple venues within the Congress for 
discussing issues are one of the strongest attributes of our democ- 
racy, even though the complexity it creates is sometimes frus- 
trating. 

In conclusion, I suggest that this Subcommittee and the Congress 
avoid attempting to adopt the PART process and instead, by focus- 
ing on accountability and results, emphasize the existing resources 
that are unique to the Legislative Branch. Instead of searching for 
a one-size-fits-all approach, the Congress has rich resources within 
the appropriations and authorizing committees structure that could 
be used to craft definitions of results within the framework of spe- 
cific programs. Performance can best be handled within the con- 
fines of specific program development and traditional Congres- 
sional oversight. 

The Congress has oversight capacity that can be used to provide 
more robust information than that from the PART process. The 
oversight process gives the Congress access to a range of informa- 
tion from GAO, from CBO, CRS, and the Inspectors General, as 
well as from non-governmental sources. Each of these sources has 
a somewhat different perspective, but collectively, they offer a rich 
view of program performance. Congress has the ability to develop 
a regularly-scheduled assessment of programs within its oversight 
role. 

Thank you for inviting me to testify, and I am available to work 
with the Subcommittee and its staff to continue this conversation. 
Thank you. 

Senator Coburn. Thank you both for your testimony. 

I know that the oversight is there. The Congress, less than 7 per- 
cent of their hearings are oversight. Ms. Radin, my question is, 
how are you going to change that? How are you going to get Con- 
gress to do the oversight that is necessary and how are we going 
to measure performance if Congress won’t do the oversight? 

Ms. Radin. Has there been an attempt to try to prop up the proc- 
ess? 

Senator Coburn. That is one of the things that Senator Carper 
and I are doing with this very Subcommittee, but it is a new at- 
tempt. We are going to average about two Subcommittee hearings 
a week. That is about 60 or 70, maybe even 80 hearings on over- 
sight. 

But the fact is, if you look at all the hearings in Congress, the 
vast majority of them are not oversight hearings. The vast number 
are on the basis of new legislation or appropriations. So my ques- 
tion to you is, if we are not going to use the PART performance 
tool, which everybody recognizes it has weaknesses, what tool are 
we going to use? 
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Let us also assume all the programs out there are good. Let us 
just make that assumption from it. They are all good, but we are 
going to come up with this budget hurdle we are going to face. So 
the thing we have to know is how do we prioritize them? Which 
is the best, because some are going to go away. I promise you, in 
the next 10 years, a large portion of the Federal Government is 
going to go away because we will not be able to afford it. So how 
do we measure what do we give best to the Federal Government? 

Ms. Radin. The Subcommittee certainly can’t do everything on 
its own. But can’t there be some effort in the organization of the 
Senate and in the House to really focus on the authorizing commit- 
tees, because they are the ones that really know the programs. It 
seems to me that there has not been a priority given to oversight 
in those committees, and that seems to me something that the or- 
ganizers of the Congress can push. 

It is not going to happen all at once. We know that there have 
been attempts to try to rationalize the budget process and it is 
maybe a little bit less irrational than it was in the past, but the 
system is so complex that thinking that we can deal with it in one 
fell swoop is just not realistic. 

Senator Coburn. With all due respect, I don’t think we are 
thinking we can do it in one fell swoop. We have seen a transition 
process. And the PART may not be the answer, but some measure 
of performance and some level of accountability so that people un- 
derstand what the end goal is that is associated with a relook at 
how you are performing on that end goal has to be a component 
of every Federal Government program that we have. We have to 
start asking the hard questions because we are going to get the 
hard questions asked about the end, the tough ones in the next 5 
to 10 years. 

Ms. Radin. I am not arguing that you shouldn’t ask those ques- 
tions. I am suggesting that Congress has to grapple with that in 
its own terms and that turning it over to 0MB really is violating 
what the Constitution has created. The ball is in your court. 

Senator Coburn. That is exactly why we are having the hearing. 
We haven’t turned it over to 0MB. But a measurement of perform- 
ance, no matter who does it, still gives some information with 
which the Congress can act on. And the question may not be 
whether Senator Carper and I agree on a program. The question 
may be which of the following ten programs, two out of them are 
going to have to go if we are going to live within the constraints 
for our grandchildren. Which two, and how do we measure that? 

To have a ratings program or an assessment program within the 
agencies within the program in terms of creating the expectation 
for performance is not a bad idea. I don’t think that there is any- 
body up here that is suggesting Congress is about to give that over. 
They are not, because the only way you change it is through Con- 
gress. 

The other point that I would just ask is you recognize that over 
25 percent of our discretionary budget is unauthorized right now. 
There is no expertise on it because there is no authorizing lan- 
guage and hasn’t been for 10 or 15 years. So Congress has its own 
problems in terms of authorizing the spending that we have. 
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First of all, unanimous consent to put the Mercatus Center eval- 
uation into the record, and without objection, that is so ordered. ^ 

Senator Coburn. Ms. Norcross, compare the Mercatus Center to 
the evaluation by 0MB and their process. If you line those two up, 
what did you see? 

Ms. Norcross. The annual scorecard? 

Senator Coburn. Yes. 

Ms. Norcross. Our annual scorecard evaluates performance, the 
annual performance reports that agencies must submit. And what 
we found this year was that about 11 percent of the budget is rep- 
resented by good reporting, and what we mean by that is that 
these agencies receive a satisfactory score according to our criteria, 
which assess reports on whether they produce public benefits, how 
transparent they are, and on leadership. 

The PART tool assesses government programs. So the annual 
scorecard we produce really evaluates a GPRA requirement. The 
PART tool evaluates on the program level. 

Senator Coburn. OK. Thank you. Senator Carper. 

Senator Carper. Let me just start off by welcoming you and 
thank you very much for being here and for the thought you have 
given to these issues and your counsel to us today and to the Con- 
gress. 

Let me start off by asking, where do you think the two of you 
agree in terms of your advice to us and where do you disagree? 

Ms. Radin. Well, I think we both agree that assessing perform- 
ance is important, that the question is how do you do it and wheth- 
er you acknowledge that there is incredible diversity of programs 
in the Federal portfolio. 

I am concerned about the PART evaluations that I think have 
really ignored the will of Congress. David Walker talked about the 
importance of Congress defining goals. Now, some of the goals and 
programs are outcome goals, but some of them are also process 
goals and those are legitimate. And yet the approach that we have 
used in both GPRA and in PART have suggested that process goals 
aren’t important. 

So, for example. Congress may develop a goal that says we want 
to involve particular groups in the decisionmaking process who 
have not been involved before. That goal is not new. The Coopera- 
tive Extension Service back in the 1930s really created process 
goals, and I think that is a very legitimate role for Congress. 

We have also seen efforts in research, for example, that are real- 
ly process goals and not outcome goals. We focus on process be- 
cause we don’t know what is going to occur as a result of a re- 
search effort. 

In other cases. Congress has actually determined the means for 
developing particular programs. In both the Clean Water Revolving 
Fund and the Consumer Product Safety Commission, Congress told 
the agency not to use cost-benefit analysis in making their deci- 
sions about programs, yet those agencies got very low ratings be- 
cause they didn’t have a cost-benefit analysis. 


^ “An Analysis of the Office of Management and Budget’s Program Assessment Rating Tool 
(PART),” June 2005, by Eileen C. Norcross, Mercatus Center, George Mason University, appears 
in the Appendix on page 87. 
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Now, you asked earlier whether 0MB, there were times in which 
0MB, in a sense, preempted the voice of Congress, and it has hap- 
pened. In another example, the Appalachian Regional Commission 
got a low rating because it was playing a coordination role and 
wasn’t outcome-oriented. But the very fact of that Appalachian Re- 
gional Commission is to focus on coordination. That is its goal. But 
it is not an outcome goal, it is a process goal. 

So I think this makes things a lot more difficult for the agency 
to fit into the PART process. 

Senator Carper. Let me go back to my original question, and I 
will direct it this time to Ms. Norcross. Where do you see you and 
Dr. Radin agreeing in your testimony? Where do you see you dis- 
agreeing? 

Ms. Norcross. I would say we both agree on the importance of 
using performance information. We might have disagreements on 
the PART itself and the extent to which it should be used by Con- 
gress. I don’t know that I advocate PART being used by Congress 
wholesale. I think the Executive Branch took the initiative to de- 
velop a tool that would evaluate the government on a program level 
and I would like to see Congress also incorporating that kind of 
performance information, whether it is the PART or not, once Con- 
gress demands performance information, agencies have to produce 
it. They have to know how their programs are doing and they have 
to produce better data, and it just gives us more objective data to 
make better decisions. 

Senator Carper. Early this morning, on the way down on the 
train, I had a telephone conference call with my State director and 
with the person who is in charge of our largest county for me. We 
were talking about the results of a faith-based housing initiative, 
a home ownership initiative that we have going in each of our 
three counties, and we were talking about how the program was 
going with respect to being successful or not. 

And the question I asked, and I ask this question a lot of my 
staff, is how do we measure success? And the response that came 
back was that we measure success with respect to how many peo- 
ple are involved in home ownership counseling programs. And I 
said, is that really the way we want to measure success, or do we 
want to measure success with respect to the number of people who 
become homeowners because of their participation in this process. 

I am sort of reminded of that conversation here today, Mr. Chair- 
man, because how we measure success is really important as to 
how we evaluate these programs. 

First of all, we have to answer that question. How do we meas- 
ure success, whether it is a home ownership program or whether 
it is a program to get people off of welfare or any variety of pro- 
grams to make us safer, but to be able to determine how we meas- 
ure success is just critical and I don’t know that we spell that out 
all that often in legislation that we pass. Since we don’t spell it out, 
I think it makes it all the more difficult for 0MB or anybody else 
to come in and say programs are a success or are doing what they 
are supposed to or not because we have not ourselves said, in order 
for this program to be successful, this has to happen. 

Ms. Radin. I would say that what you are describing is a reliance 
on output measures rather than outcomes. One of the reasons why 
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we tend to fall back on output measures is because so many of the 
outcomes are not measurable today but in the future. Yet we are 
talking about annual budget processes. You have to give a pro- 
gram, let us say, 5 years or 10 years to really show any outcomes. 
So we use the outputs as an indicator, and this is a classic problem 
for the whole evaluation field. People have really tried to figure out 
how you can focus on outcomes when you have a time delay in the 
process. 

Senator Carper. All right. Thank you. 

Senator Coburn. Well, the important thing is that we have some 
process ongoing to measure, and it is not just that we are going to 
measure it, it is that we are going to create the expectation that 
it is going to be measured, which is just as important as the meas- 
urement itself. 

Just a couple of things. First of all, I thank each of you for com- 
ing. I have not gotten through the Center’s evaluation. I am going 
to do that. I would like to leave the record open so that we can 
send you additional questions. 

I guess one additional question for you. Dr. Radin. Can you not 
measure process? In other words, if the Appalachian Regional Com- 
mission is a coordinator of effort, can you not measure that coordi- 
nation? 

Ms. Radin. You can, but the way that we have approached these 
kinds of processes 

Senator Coburn. Now you are talking about OMB’s approach. 

Ms. Radin. Yes. 

Senator Coburn. OK. 

Ms. Radin. I think you can measure processes as long as you ac- 
knowledge that the process is what Congress is trying to accom- 
plish with the program. That will be different from outcomes. 

Senator Coburn. I think that is true, and I would assure you 
with the diversity in the Senate that what 0MB brings to us, we 
are going to look at that evaluation and then we are going to make 
an independent judgment on what is or what isn’t going to be fund- 
ed — and unfortunately, from my viewpoint, a lot of things are going 
to get funded that shouldn’t and a lot of things may not get funded 
that should, and just the opposite viewpoint on somebody sitting on 
a different political spectrum. 

But the most important thing is to reach above that and say, no 
matter whether we agree or not, let us say we agree on all of them 
needing to be funded. How are we going to measure which ones 
have to have priority, because that is really what is coming. What 
is the priority for the Federal Government in how we spend, what 
we spend, and where do we spend it? 

We are not going to see the polarization on issues, I think, in the 
future because the problems economically are going to be so dif- 
ficult. It is going to be which ones have the highest priority and 
which ones do we as a group think needs to be funded first, second, 
third, fourth, and fifth. So measurement of whether it is outcome 
or output or trend lines on outcome or output are going to become 
very valuable for us. 

Ms. Radin. One of the things that I have been concerned about 
is that I think most of the effort in the performance area (not just 
PART), has really focused on efficiency values. But there also are 
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effectiveness values and there are also equity values and we 
shouldn’t forget them because many programs have all of those ele- 
ments. 

Senator Coburn. And I think the other thing that we heard be- 
fore you came is that there are some programs that aren’t per- 
forming because they are underfunded. In other words, they are 
not going to achieve the critical mass to achieve the goal that Con- 
gress wanted them to because the resources haven’t been put there. 
So we need to look at all of that. 

Any other comments either of you might want to make? 

Senator Carper. Mr. Chairman, I do have one last question, if 
I could. This discussion really reminds me of conversations I have 
had with my colleagues before about programs that are not being 
authorized, not being reauthorized, not being revisited, and yet the 
funding continues. In some cases, it makes sense. In other cases, 
frankly, it does not. In some cases, it reflects not an inappropriate 
thing to go forward and to fund a program that has not been reau- 
thorized. In other cases, I think it really reflects a failure on our 
part in the Legislative Branch. 

I would like to ask Dr. Radin, one last question, if you don’t 
mind. I think you argue, I believe in your testimony, that at least 
some of the ratings given by 0MB reflect some of the views that 
critics have held about programs for some time. Are there weak- 
nesses within the Program Assessment Rating Tool or in the way 
it has been implemented that allow or maybe even encourage bias? 

And as a follow-up, do you think it is possible for 0MB, regard- 
less of whether it is staffed by Democrats or by Republicans, is it 
possible for 0MB to do the kind of work and make the kind of de- 
terminations that the rating tool calls for without introducing some 
level of bias, whether it be political or ideological? 

Ms. Radin. I think one of the points that GAO made in its report 
a year ago January was that there is incredible variability in 0MB 
in the way the budget examiners have been dealing with PART. 
And so you have some cases in which a budget examiner for the 
last 20 years has been trying to kill a program, makes a rec- 
ommendation for zero budgeting and Congress puts the money 
back. And then there are other cases in which the budget examiner 
loves the program and so it does extremely well. 

And so I think that this process really gives an incredible 
amount of authority to the individual budget examiner. Much of 
that is not really transparent so that we don’t know why a rating 
occurs. The child welfare community-based Child Abuse Prevention 
Program was rated in a number of elements that, “this element is 
not applicable.” Yet the overall rating was “results not dem- 
onstrated” and nobody really quite understands how you got from 
that, the “not applicable” to the “results not demonstrated.” 

So the fact that this isn’t transparent, I think is a big issue, and 
that is why this process is too important to really have it central- 
ized in the hands, really, of a small number of people in 0MB. 

Senator Carper. All right. Thanks to both of you very much. 

Senator Coburn. Thank you. The meeting is adjourned. 

[Whereupon, at 3:40 p.m., the Subcommittee was adjourned.] 
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What GAO Found 

The federal government is in a period of profound transition and faces an 
array of chaUenges and opportunities to enhance performance, ensure 
accountabihty, and petition the nation for the future. A number of 
overarching trends — ^including the nation’s long-term fiscal imbalance — 
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sustained. Such an effort can strengthen the budget process itself and 
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Mr. Ch^rman and Members of the Subcommittee: 

I am pleased to be here today to discuss performance budgeting and the 
Office of Mant^ement ^d Budget’s (0MB) Program Assessment Rating 
’Tbo] (PART).* Given current trends and challenges facing the nation- 
including the federal government’s long-term fiscal imbalance — it is critical 
to reexamine the relev-ancy of federal programs and their fit with national 
priorities, while maximizing program performance within current and 
expected resource levels. The implementation of performance budgeting 
approaches can be an important step to help achieve this goal. 

As Congress is well aware, our nation is currently on an unsustainable 
fiscal path. Long-term budget simulations by GAO,^ the Congressional 
Budget Office (CBO), and others show that we face a large and growing 
long-term structural deficit due primarily to known demographic trends 
and rising health care costs. Continuing on this unsustainable fiscal path 
will gradually erode, if not suddenly damage, our economy, our standard of 
living, and ultimately our national security. All reasonable simulations 
indicate that the problem is too big to be solved by economic growth alone 
or by making modest changes to existing spending and tax policies. Rather, 
a fundamental reexamination of major spending and tax policies and 
priorities will be important to recapture our fiscal flexibility and ensure 
that our programs and priorities respond to key emerging social, economic, 
and security changes and challenges. 

Performance budgeting holds promise as part of a process of reexamining 
the base of the federal government. It can help enhance the government’s 
capacity to assess competing claims for federal dollars by arming decision 
makers with better information both on the results of individual programs 
as weD as on entire portfolios of policies, programs, and other tools 
designed to address common goals. However, it is also important to 
remember that in a political process performance information is likely to 
be one, but not the only, factor in budgetary decision making. In other 
words, performance information can change the terms of debate but it will 
not necessarily deteimine the ultimate decision. 


‘In this testiinony, the term performance budgeting refers to any linkage between budgeting 
and expected or actual evidence-based performance information. 

*Por more information see GAO’s Web site “Our Nation’s Fiscal Outlook: The Federal 
Government’s Long-Term Budget Imbalance,” http://www, gao.gov/speclal, pubs/longterm. 
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Existing performance budgeting efforts, such as the Government 
Performance and Results Act of 1993 (GPRA) ® and PART, provide a 
foundation for a baseline review of existing federal policies, programs, 
functions, and activities. We recognize that this will not; be easy. Reforming 
programs and ^irtivities leads to winners and losers, notwithstanding 
demonstratwi shortfalls in performance and design. Given prior 
experiences and poUtictd realities, there is little real “low-hanging fruit” in 
the federal budget 

Today I will touch fimt on the need for a fundamental reexamination of 
government given our nation’s long-term fiscal challenge. Then I will turn 
to and discuss the import^mt role of performance budgeting in any such 
reexamination. The successful ^plication of performmce budgeting in 
this reexamination process rests on 

• continuing to build on die legaty of GPRA by improving the reliability 
and credibility of performance information and increasing program 
evaluation capacity; 

• encouraging demand for that information by garnering stakeholder buy- 
in— particularly from Congress — on what to measure and how to 
present this information, since only then will it be linked to the 
congressional authorization, appropriations, and oversight processes; 
and 

• developing a comprehensive, crosscutting approach to assessing the 
performance of all programs — including tax expenditures — relevant to 
common goals. 

This testimony draws upon our wide-ranging work on GPRA and federal 
budget and performance integration and on information in the President’s 
Budget of the U.S. Govemmetit, Fiscal Year 2006, specifically the budget 
and performance integration initiative of the President’s Management 
Agenda (PMA). We conducted our work in accordance with generally 
accepted government auditing standards. 


‘’Pub. L No. 103-62 (1993). 
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Long-term Fiscal 
Challenge Provides 
Reexamination 
Impetus 


As I noted, known demognq>hic trends and rising health care costs are 
in^or drivers of the nation’s large and growing structural deficits. The 
nation cannot ignore this fiscal pressure — it is not a matter of whether the 
nation deals with the filial gap, but how and when. GAO’s long-term budget 
simulations illustrate the magnitude of this fiscal challenge. Figures 1 and 2 
show these simulations under two different sets of assumptions. Figure 1 
uses the CBO January 2(X)5 baseline through 2015. As required by law, that 
baseline assume no changes in current law, that discretionary spending 
grows with inflation through 2015, and that all tax cuts currently scheduled 
to expire are permitted to expire. In Figure 2, two assumptions about that 
first 10 years are changed: (1) discretionary spending grows with the 
economy rather than with inflation and (2) all tax cuts currently scheduled 
to expire are made permanent. In both simulations discretionary spending 
is affiumed to grow with the economy after 2015 and revenue is held 
constant as a share of Gross Domestic Product (GDP) at the 2015 level. 
Also in both simulations long-term Social Security and Medicare spending 
are based on the 2005 trustee’s intermediate projections, and we assume 
that benefits continue to be paid in full after the trust funds are exhausted. 
Long-term Medicaid spending is based on CBO’s December 2003 long-term 
projections under midrange assumptions. 
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Figure 1 : Compositim of Spending as a Share of GOP under Baseline Extended 
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Note: in addition to the expiration of tax cuts, revenue as a share of GDP increases through 201 6 due 
to (1 ) real bracket creep, (2) more taxpayers becoming subject to the Alternative Minimum Tax (AMT), 
and (3) increased revenue from tax-deferred retirement accounts. After 201 5. revenue as a share of 
GOP is held constant. 
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Figure 2: Composition of Spending as a Share of GDP Assuming Discretionary 
Spending Grows with GDP after 2005 and Ail Expiring Tax Provisions Are Extended 
Percent of 
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Note; Although expiring tax provisions are extended, revenue as a share of GDP Increases through 
2015 due to (1 ) real bracket creep, (2) more taxpayers becoming subject to the AMT, and (3) Increased 
revenue from tax-deferred retirement accounts. After 201 5, 'revenue as a share of GDP is held 
constant. 


As these simulJdions illustrate, absent policy changes on the spending 
and/or revenue side of the budget, the growth in spending on federal 
retirement and health entitlements will encumber an escalating share of the 
government’s resources. Indeed, when we assume that recent tax 
reductions are made permanent and discretionary spending keeps pace 
with the economy, our long-term simulations suggest that by 2040 federal 
revenues may be adequate to pay little more than interest on the federal 
debt. Neither slowing the growth in discretionary spending nor allowing 
the tax provisions to expire~nor both together— would eliminate the 
imbalance. Although federal tax policies will likely be part of any debate 
about our fiscal future, making no changes to Social Security, Medicare, 
Medicaid, and other drivers of the long-term fiscal gap would require at 
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least a doubling of federal taxes in the future — and that seems both 
unrealistic and inappropriate. 

These challenges would be difficult enough if all we had to do is fund 
existing conunitments. But as the nation continues to change in 
fundamental ways, a wide range of emerging needs and demands can be 
expected to compete for a share of the budget pie. Whether national 
security, transportation, education, or public health, a growing population 
will generate new claims for federal actions on both the spending and tax 
sides of the budget 

Although demographic shifts and rising health care costs drive the long- 
term fiscal outlook, they are not the only forces at work that require tire 
federal government to rethink its role and entire approach to policy design, 
priorities, and management Other important forces are working to reshape 
American society, our place in the world, and the role of the federal 
government These include evolving defense and homeland security 
policies, increasing global interdependence, and advances in science and 
technology. In addition, the federal government increasingly relies on new 
networks tmd partnerships to achieve critical results and develop public 
policy, often including multiple federal agencies, domestic and 
international non- or quasi-government organizations, for-profit and not- 
for-profit contractors, and state and local governments. If government is to 
effectively address these trends, it cannot accept all of its existing 
programs, policies, and activities as “givens." Many of our programs were 
designed decades ago to address earlier challenges. Outmoded 
commitments and operations constitute an encumbrance on the future that 
can erode the capacity of the nation to better align its government with the 
needs and demands of a changing world and society. Accordingly, 
reexamining the base of all major existing federal spending and tax 
programs, policies, and activities by reviewing their results and testing 
their continued relevance and relative priority for our changing society is 
an important step in the process of assuring fiscal responsibility and 
facilitating national renewal.'* 

A periodic reexamination offers the prospect of addressing emerging needs 
by weeding out programs and policies that are redundant, outdated, or 


■‘For more information on reexamination of federal programs, see GAO, 2V‘ Century 
Challenges: Reexamining the Base of the Federal Government, GAO-05-325SP 
(Washingt<m, D.C.: February 2{K^). 
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ineffective. Those programs and policies that remain relevant could be 
updated and modernized by improving their targeting and efficiency 
through such actions as redesigning allocation and cost-sharing provisions, 
consolidating faciliti^ and programs, and streamlining and reengineering 
operations and processes. The tax policies and programs financing the 
federal budget can be reviewed vvtth an eye toward both the overall 
level of revenues that should be raised as well as tire mix of taxes that are 
used. 

We recognize that taking a hard look at existing programs and caretuUy 
reconsidering their goals and financing are challenging tasks. Reforming 
programs and activities leads to winners and losers, notwithstanding 
demonstrated shortfalls in performance and design. Moreover, given the 
wide range of programs and issues covered, the process of rethinking 
government programs and activities may take a generation to unfold. 

We are convinced, however, that reexamining the base offers compelling 
opportunities to both redress our current and projected fiscal imbalance 
whUe better positioning government to meet the new challenges and 
opportunities of this new century. In this regard, the management and 
performance reforms enacted by Congress in the past 15 yeais have 
provided new tools to gain insight into the financial, program, and 
management performance of federal agencies and activities. The 
infoimation being produced as a result can provide a strong basis to 
support the needed review, reassessment, and reprioritization process. 


Current Performance 
Budgeting Initiatives 
Hold Promise for 
Reexamining the Base 


While this kind of oversight and reexamination is never easy, it is helped by 
the availability of credible performance information focusing on the 
outcomes achieved with budgetary resources and other tools. Performance 
budgeting can help enhance the government’s capacity to assess competing 
claims in the budget by fuming budgetary decision makers with better 
infomiation on the results of both individual programs as well as entire 
portfolios of tools and programs addressing common outcomes. To 
facilitate application of performance budgeting in reexamination, it is 
useful to understand the current landscape. Going forward, decision 
makers need a road map— grounded in lessons learned from past 
initiatives — that defines what successful performance budgeting would 
look like and identifies the key elements and potential pitfalls on the 
critical path to success. Central to this is an understanding of what is meant 
by succe^ in performance budgeting and the key factors that influence 
that success. 
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Current Performance Performance budgeting efforts are not new at the federal level. In the 

Budgeting Initiatives Are 1990s, Congress and the executive branch drew on lessons learned from 
Grounded in Past Efforts years of efforts to link resources to results to lay out a statutory and 

management framework titat provides the foundation for strengthening 
government performance and accountability. With GPRA as its centerpiece, 
these reforms also laid the foundation for perfonnance budgeting by 
establishing infrastructures in the agencies to improve the supply of 
information on performance and cc«ts. GPRA is designed to inform 
congre^ional and executive decision making by providing objective 
information on the effectiveness and efficiency of federal programs and 
spending. A key purpose of GPRA is to create closer and clearer links 
between the process of allocating scarce resources and the expected 
results to be achieved with those resources. Importantly, GPRA requires 
both a connection to the structures used in congressional budget 
presentations and coitsultation between the executive and legislative 
branches on agency strategic plans. Because these requirements are 
grounded in statute, this gives Congress an oversight stake in GPRA’s 
success.® Over a decade after its enactment, GPRA has succeeded in 
expanding the supply of performance information and institutionalizing a 
culture of perfonnance as well as providing a solid foundation for more 
recent budget and perfonnance initiatives.® In part, this success can be 
attributed to the fact that GPRA melds the best features, and avoids the 
woi^st, of its predecessors. 

Building on GPRA’s foundation, the current administration has made the 
integration of performance and budget information one of five 
governmentwide management priorities under its PMA.’ PART is central to 


^ Pub. L No. 1<XW2 § 2, 5 U.S.C. § 306, and 31 U.S.C. |§ 1115-1116. 

*GAO, Results-Ori^tfid Goxiemment: GPRA Has Established a Solid Foundation/or 
Achieving Greater RefmMs, GAO-04-38 (Washington, D.C.: Mar. 10, 2004), 

^In addition to budget and perfonnance inte^ation, the other four priorities uncier PMA are 
strategic manag^ent of human capital, expanded electronic government, improved 
financia] performance, and competitive sourcing. 
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the Administration’s budget and performance integration initiative.^ 0MB 
describes PART as a diagnostic tool meant to provide a consistent 
approach to assessing federal programs as part of tiie executive budget 
formulation proce^. It ^plies 25 questions to all “programs”** under four 
broad topics; (1) program purpose and design, (2) strategic planning, 

(3) program mwagement, and (4) program results (i.e., whether a program 
is meeting its long-term and annual goals) as well as additional questions 
that are specific to one of seven mechanisnrs or approaches used to deliver 
iheprogram.^® 

Drawing on available performance and evaluation information, the PART 
questionnaire attempts to determine the strengths and weaknesses of 
federal programs with a particular focus on individual program results and 
improving outcome measures. PART asks, for example, whether a 
program’s long-term goals are specific, ambitious, and focused on 
outcomes, and whether annujd goals demonstrate progress toward 
achieving long-term go^s. It is designed to be evidence-based, drawing on a 
wide array of infonnation, including authorizing legislation, GPRA strategic 
plans and performance plans and reports, financial statements, inspector 
general Mid GAO reports, and independent program evaluations. 

Since die fiscal year 2004 budget cycle, 0MB has applied PART to 607 
programs (about 60 percent of the federal budget) and given each program 
one of four overall ratings: (1) “effective,” (2) “moderately effective,” 

(3) “adequate," or (4) “ineffective” based on program design, strategic 
planning, man^ement, and results. A fifth rating, “results not 
demonstrated,” was given — independent of a program’s numerical score — 


"For a detailed examination of PART, see GAO, Performance Budgeting: Observations on 
the Use of OMD’s l^gratn Assessment Rating Iholfcrr the Fiscal Year 2004 Budget, GAO- 
04-174 (Washington, D.C.; Jan. 30, 2004). Another significant element of the performance 
and budget integration initiative is efforts to restructure budgets, See GAO, Performance 
Budgeting: Efforts to Restructure Budgets to Better Align Resources with. Performance, 
GAO-(^l 17SP (Washington, D.C.: February 2005). 

^ere is no standard definition for the tenn "program,” For purposes of PAST, 0MB 
described the imit of analysis (program) as (1) an activity or set of activities clearly 
recognized as a program by the public, 0MB, and/or Congress; (2) having a discrete level of 
funding clearly associated with it; and (3) corresponding to the level at which budget 
decisions are made. 

‘The seven n^jor categories are competitive grants, block/formula grants, capital assets 
and service acquisition programs, credit programs, regulatory-based programs, direct 
federal programs, and research and development programs. 
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if 0MB decided that a program’s performance information, performance 
measures, or both were insufficient or inadequate. During the next 2 years, 
the Administration plans to assess all remaining executive branch 
programs with limited exceptions.^‘ 

As I testified before tiiis subcommittee in April, PMA and its related 
initiatives, including PART, demonstrate the Administration’s commitment 
to improvii^ federal management and performance. By calling attention to 
successes and needed improvements, the focus that these initiatives bring 
is certainly a step in the right direction, and our work shows that progress 
has been made in several important areas over the past several years. 
However, it is not clear that PART has had any significant impact on 
congressional authorization, appropriations, and oversight activities to 
date. In order for such efforts to hold appeal beyond the executive branch, 
developing credible performance information and garnering congressional 
buy-in on what to measure and how to present this information to them are 
critical. Otherwise, as some congressional subcommittees have noted, 
PART is unlikely to play a m^or role in the authorization, appropriations, 
and oversight processes. 

Prior initiatives have left us with some lessons about how to build a 
sustainable approach to linking resources to results. Before I discuss those 
critical factors let me touch briefly on the importance of realistic 
expectations. I say this because previous management reforms have been 
doomed by inflated and unrealistic expectations. Performance budgeting 
can do a great deal: it can help policymakers address important questions 
such as whether programs are contributing to their stated goals, are well- 
coordinated with related initiatives at the federal level or elsewhere, tmd 
are targeted to the intended beneficiaries. However, it should not be 
expected to provide the answers to all resource allocation questions in 
some automatic or formula-driven process, Performance problems may 
well prompt budget cuts, program consolidations, or eliminations, but they 
may also inspire enhanced investments and reforms in program design and 
management if the program is deemed to be of sufficiently high priority to 
the nation. Conveisely, even a program that is found to be exceeding its 


“The administration is considering alternative methods and timelines for assessment of 
progrante with limited impact and large activ-ities where it is difficult to determine an 
iqipropriate unit of analysis. 

'^GAO, Management Reform: Assessing the Pi-esident's Management Agenda, GAOOS- 
574T (Washington. D.C.; Apr. 21. 2005). 
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performance expectations can be a candidate for budgetary cuts if it is a 
lower priority than otiter competing claims in the process. The 
determination of priorities is a function of competing values and interests 
that may be informed by performance information but also reflects other 
factors, such as the overall budget situation, the state of the economy, 
security needs, equity considerations, unmet societal needs, and the 
appropriate role of the federal govermnent in addressing any such needs. 

Accordingly, we found that while PART scores for fiscal year 2004 were 
generally positively related to the Administration’s proposed funding 
changes in discretionary programs, the scores did not automatically 
determine funding changes. That is, for some programs rated “effective” or 
“moderately effective” 0MB recommended funding decreases, while for 
several programs judged to be “ineffective” 0MB recommended additional 
funding in the President’s budget request with which to implement 
changes.^ As we have noted, success in performance budgeting should not 
be defined only by its impact on funding decisions but also on the extent to 
which it helps inform Congress and executive branch policy decisions and 
improve program management.^ In this regard, for the fiscal year 2004 
PART assessments we reported that over 80 percent of the PART 
recommendations focused on improving program management, 
assessment, and design; less thmi 20 percent related to funding.^® 

We also reported that OMB’s ability to use PART to identity and address 
future program improvements and measure progress — a major purpose of 
PART“is predicated on its ability to oversee the implementation of PART 
recommendations. At the request of the Chairman of the House 
Subcommittee on Government Management, Finance, and Accoimtability, 
Committee on Government Reform, we are currently conducting a review 
of (1) OMB's and agencies' perspectives on the effects PART 
recommendations are having on agency operations and results and issues 
encountered in responding to PART recommendations; (2) OMB’s 
leadership and direefion in ensuring an integrated, complementary 
relationship between PART and GPRA, including how 0MB is assessing 
performance when multiple programs or agencies are involved in meeting 


”GAO-04-174, 14. 

"GAO, The Government Performance and Results Act: 1997 Governmenhuide 
Implementation Will Be Uneven, GAO/GGD-97-109 (Washington, D.C.: June 2, 1997), 90. 

'^GAO-04-174, 12. 
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goals a«d objectives; and (3) steps 0MB has taken to involve Congress in 
the PART process. 


Achieving Success in 
Performance 
Budgeting Requires 
Credible Information, 
Congressional “Buy- 
in,” and a 

Comprehensive and 

Crosscutting 

Perspective 


me now turn to three factors we believe are critical to sustaining 
successful performance budgeting over time: 

1. building a supply of credible performance information, 

2. encouraging demand for that information and its use in congressional 
processes by gamering stakeholder buy-in, and 

3. taking a comprehensive and crosscutting approach to assessing related 
programs and policies. 


Having a Supply of Credible Tlte credibility of performance information, including related cost data, and 
Performance Information ability of federal agencies to produce credible evaluations of their 

programs’ effectiveness are key to the success of perfomtance budgeting, 
As I testified before this subcommittee in April, this type of information is 
critical for effective performance measurement to support decisions in 
areas ranging from program efficiency and effectiveness to sourcing and 
contract management. To be effective, this information must not only be 
timely and reliable, but also both useful and used. Agencies are expected to 
implement integrated financial and performance management systems that 
routinely produce information that is (1) timely — to measure and affect 
performance, (2) useful — to make more informed operational and investing 
decisions, and (3) reliable — to ensure consistent and comparable trend 
analysis over time and to facilitate better performance measurement and 
decision making. Producing timely, useful, and reliable information is 
critical for achieving the goals that Congress established in GPRA, the 
Chief Financial Officers (CFO) Act of 1990, and other federal financial 
management reform legislation. 


“Pub. L No. 101-576(1990). 
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Unfortunately, as our work on PART and GPRA implementation shows, the 
credibility of perfomiance data has been a long-standing weakness.’^ 
Likewise, our work has noted limitations in the quality of agency evaluation 
infonnation and in agency capacity to produce rigorous evaluations of 
program effectiveness. We have previously reported that agencies have had 
difficulty assessing many program outcomes that are not quickly achieved 
or readily observed and contributions to outcomes that are only partly 
influenced by federal funds.*® Furthermore, our work has shown that few 
agencies deployed the rigorous research methods required to attribute 
changes in undertyir^ outcomes to program activities.*^ Our 2003 review of 
agencies’ evaluation capacity identified four main elements that can be 
used to develop and improve evaluation efforts. They are (1) an evaluation 
culture, (2) data quality, (3) analytic expertise, and (4) collaborative 
partnerships.®* 

0MB, through its development and use of PART, has provided agencies 
with a powerful incentive for improving data quality and availability. 
Agencies may make greater investments in improving their edacity to 
produce and procure quality information if agency program managers 
perceive that program performance and evaluation data will be used to 
make actual resource decisions throughout the resource allocation process 
and can help them get better results. 

Improvements in the quality of performance data and the capacity of 
federal agencies to perform program evaluations will require sustained 
commitment and investment of resources. Over the longer term, failing to 
discover and correct performance problems can be much more costly. 
More importantly, it is critical that budgetary investments in this area be 
viewed as part of a broader initiative to improve the accountability and 
management capacity of federal agencies and programs. 


' 'GAO has suggested various approaches to addressing this and other challenges. See 
GAO/GGD-97-109 and GA04)4-38. 

'®GAO, Performance Budgeting: Opportunities and Challenges, GAO-02-n 06T 
(Washington, D.C.: Sept 19, 2002). 

‘*GAO, Program Evalua tion: Agencies Qiailenged by New Demand for Information on 
Program Results, GAO/GGD-98-.53 (Washington, D.C.: Apr. 24, 1998). 

*GAO, Program Evaluation: An EvatuatUm Culture and Collaborative Partnerships Help 
Build Agency Capacity, GAO-03454 (Washington, D.C.. May 2, 2003). 
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Obtaining Congressional 
Buy-in 


Federal performance and account^ility refonns have given much 
attention to increasing tiie supply of performance infonnation over the past 
severe decades. However, improving the supply of performance 
infonnation is in and of itself insufficient to sustain performance 
management and achieve real improvements in management and program 
results. Rather, it needs to be accompanied by a demand for and use of that 
infonnation by decision makers and managers alike. Key stakeholder 
outreach and involvement is critical to building demand and, therefore, 
success in performance budgeting. 

Lack of consensus by a community of interested parties on goals and 
measures and the way that they are presented can detract from the 
credibility of performance information and, subsequently, its use. Fifty 
years of past executive branch efforts to link resources with results have 
shown that any succ^sful effort must involve Congress as a full partner. 
We have previously reported that past performance budgeting initiatives 
faltered in large part because they intentionally attempted to develop 
performance plans and measures in isolation from the congressional 
authorization, appropriations, and oversight processes.^^ \^ile 
congressional buy-in is critical to sustain any major management initiative, 
it is especially important for performance budgeting given Congress's 
constitutional role in setting national priorities and allocating the resources 
to achieve them. 

Obtaining buy-in on goals and measures from a community of interested 
parties is critical to facilitating use of performance information in resource 
allocation decisions. PART was designed for and is used in the executive 
branch budget preparation and review process; as such, the goals and 
measures used in PART must meet OMB’s needs. However, the current 
statutory framework for strategic planning and reporting is GPRA — a 
broader process involving the development of strategic and performance 
goals and objectives to be reported in strategic and annual plans. OMB’s 
desire to collect performance data that better align with budget decision 
xinits means that the fiscal year 2004 PART process became a parallel 
competing structure to the GPRA framework. Although 0MB 
acknowledges th^ GPRA was the starting point for PART, the emphasis Ls 
shifting. Over time, as the performance measures developed for PART are 


^'GAO, Performance Budgeting: Past Initiatives Offer Insights for GPRA Imple7nen.tation, 
GAO/AIMD-97-46 (Washington, D.C.: Mar. 27. 1997). 
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used in the executive budget process, these measures may come to drive 
agencies’ strategic planning processes. 

Opportunities exist to strengthen PART’S integration with the broader 
GPRA planiung process. Some tension about the amount of stakeholder 
involvement in the internal deliberations surrounding the development of 
PART measures and the broader consultations more common to the GPRA 
strategic plannii^ process is inevitable. Compared to the relatively open- 
ended GPRA process, any budget formulation process is likely to seem 
closed. Howewr, if PART is to be accepted as other than one element In the 
development of the President’s budget proposal, congressional 
understanding and acceptance of the tool and its analysis will be critical. 

As part of the executive branch budget formulation process, PART must 
clearly serve the President’s interests. However, measures developed solely 
by the executive branch for the purposes of executive budget formulation 
may discourage their use in other processes, such as internal agency 
management and the congressional budget process, especially if measures 
that serve these other processes are eliminated through the PART process. 
PART’S focus on outcome measures may ignore stakeholders’ needs for 
other types of measures, such as output and workload information. Our 
recent work examining performance budgeting efforts at both the state and 
federal levels revealed that ^propriations committees consider workload 
and output measures important for making resource allocation decisions.^® 
Workload and output measures lend themselves to the budget process 
because workload measures, in combination with cost-per-unit 
information, can be used to help develop appropriation levels and 
legislators can more easily relate output information to a funding level to 
help define or support a desired level of service. Like PART, GPRA states a 
preference for outcome measures. However, in practice, GPRA also 
recognizes the need to develop a range of measures, including output and 
process measures. Since different stakeholders have different needs and no 
one set of goals and measures can serve all purposes, PART can and should 
complement GPRA but should not replace it. 

Moreover, as we have previously reported, several appropriations 
subcommittees have cited the need to link PART with congressional 


QAQ, Performance Btulgexirig: Slates' ExperienccR Cm Inform Federal Efforts, GAO- 
0^215 (Washington, D.C.; Fd). 28, 2005) and GAO-05 liTSP, 
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oversi^t^ For example, the House Report accompanying the 
TVansportation and TVeasury Appropriations BUI for fiscal year 2004 
included a statement in support of PART, but noted that the 
Administration’s must be linked with the oversight of Congress to 
maximize Hie utility of the PART process, and that if the Administration 
treate as privileged or confidential the details of its rating process, it is less 
likely that Congress will use those results in deciding which programs to 
fund.^ Moreover, the subcommittee said it expects 0MB to involve the 
House and Senate Committees on Appropriations in the development of tire 
PART ratings at all stages in the process. 

In our January 2004 report on PART,^ we suggested steps for both 0MB 
and Congress to take to strengthen the dialogue between executive branch 
officials and key congression^ stakeholders, and 0MB generally agreed. 
We recommended that 0MB reach out to key congressional committees 
early in the PART selection process to gain insight about which program 
areas and performance issues congressional officials consider warrant 
PAffT review. Engaging Congress early in the process may help target 
reviews with an eye toward those areas most likely to be on the agenda of 
Congress, thereby better ensuring the use of performance assessments in 
resource allocation processes throughout government. 

The importance of getting buy-in for successful performance budgeting can 
be seen in the experience of OMB's recent efforts to restructure budget 
accounts.^ While 0MB staff and ^ency officials credited budget 
restructuring with supporting results-oriented management, the budget 
changes did not meet the needs of some congressional appropriations 
committees. While congressional appropriations subcommittee staff 
expressed general support for budget and performance integration, they 
objected to changes that substituted rather than supplemented information 
traditionally used for t 4 >propriations and oversight purposes. As we said in 
our February 2005 report on this issue, the greatest challenge of budget 
restructuring may be discovering ways to reflect both the broader planning 


^H.R. Rep. No. 108-243, pp. 168-69 (2003). 

“GA(>94-274. 

“For more information on this effort, see GAO-05-1 17SP. 
^GAO-05-iI7SP. 


Page 16 


GAO-05-709T 


51 


perspective that can add value to budget deliberations and foster 
accountability in ways that Ck)ngress considers appropriate for meeting its 
authorizing, appropri^ons, and oversight objectives. 

Going forward, inhrsing a performance perspective into budget decisions 
may only be achieved when the underlying information becomes more 
credible, accepted, and used by all major decision makers. Thus, Congress 
must be considered a full partner in any efforts to infuse a performance 
budget perspective into budget structure and budget deliberations. In due 
course, once the goals and underlying data become more compelling and 
used by Congress, budget restructuring may become a more compelling 
tool to advance budget and performance integration. 


Reexamination Requires a While existing performance budgeting initiatives provide a foundation for a 
Crosscutting Perspective baseline review of federal policies, programs, functions, and activities, 

several changes are in order to support the type of reexamination needed. 
For example, PART focuses on individual programs, but key outcome- 
oriented performance goals — ranging from low income housing to food 
safe^ to counterterrorism — are addressed by a wide range of 
discretionary, entitlement, tax, and regulatory approaches that cut across a 
number of agencies. While PARTs program-by-program approach fits with 
OMB’s agency-by-agency budget reviews, it is not well suited to addressing 
crosscutting issues or to looking at broad program areas in which several 
programs address a common goal. 

The evaluation of programs in isolation may be revealing, but a broader 
perspective is necessary for an effeedve overall reexamination effort. It is 
often critical to understand how each program fits with a broader portfolio 
of tools and strategies — such as regulations, direct loams, and tax 
expenditures — to accomplish federal missions and performance goals. 
Such an analysis is necessary to capture whether a program complements 
and supports other related programs, whether it is duplicative and 
redundant, or whether it actually works at cross-purposes to other 
initiatives. 0MB reported on a few crosscutting PART assessments in the 
fiscal year 2006 budget and plans to conduct additional crosscutting 
reviews In 2005. However, wc would urge a more comprehensive and 
consistent approach to evaluating all programs relevant to common goals. 

Such an approach would require assessing the performance of ali programs 
related to a particular goal — ^including tax expenditures and regulatory 
programs — using a common framework. Our federal tax system includes 
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hundreds of billions of dollars of annual expenditures — the same order of 
magnitude as total discretionary spending. Yet relatively little is known 
about the effectiveness of tax incentives in achieving the objectives 
intended by Congress. PART, OMB's current framework for assessing the 
performance of federal programs, has not been applied to tax expenditures. 
Assessing complete portfolios of tools related to key outcome-oriented 
goals is absolutely critical to the type of reexamination needed. The 
govemmentwide performance plan required by GPRA could help address 
this issue. 

GPRA requires the President to include in his annual budget submission a 
federal government performance plan. Congress intended that this plan 
provide a “single cohesive picture of the annual performance goals for the 
fiscal year.”® Tlie governmentwide performance plan could help Congress 
and the executive branch address critical federal performance and 
management issues, including redundancy and other inefficiencies in how 
we do business. It could also provide a framework for any restructuring 
efforts. Unfortunately, this provision has not been fully implemented. 
Instead, 0MB has used the President’s budget to present high-level 
information about agencies and certain program performance issues. The 
a^ency-by-agency focus of the budget does not provide the integrated 
perspective of government performance envisioned by GPRA. 

If the govemmentwide performance plan were fully implemented, it could 
also provide a framework for congressional oversight and other activities. 
In that regard, we have also suggested that Congress consider the need to 
develop a more systematic vehicle for communicating its top performance 
concerns and priorities; develop a more structured oversight agenda to 
prompt a more coordinated congressional perspective on crosscutting 
performance issues; and use this agenda to inform its authorization, 
appropriations, and oversight processes. One possible approach would 
involve developing a congressional performance resolution identifying the 
key oversight and performance goals that Congress wishes to set for its 
own committees and for the government as a whole. Such a resolution 
could be developed by modifying the current congressional budget 
resolution, which is already organized by budget function. Initially, this may 
involve collecting the “views and estimates” of autliorization and 
appropriations committees on priority performance issues for programs 
under their jurisdiction and working with such crosscutting committees as 


lU'p. No. 103-68, p. 27 (1993). 
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this committee, the House Committee on Government Reform, and the 
House Committee on Rules. 

In addition, we have previously recommended that Congress consider 
amending GPRA to require the President to develop a govemmentwide 
strategic plan to provide a framework to identify long-term goals and 
strategies to address issues that cut across federal agencies.^® A strategic 
plan for the federal government, supported by key national outcome-based 
indicators to assess the government’s performance, position, and progress, 
could be a valuable tool for governmentwide reexamination of existing 
programs, as well as proposals for new programs. Developing a strategic 
plan can help clarify priorities and unify stakeholders in the pursuit of 
shared goals. Therefore, developing a strategic plan for the federal 
government would be an important first step in articulating the role, goals, 
and objectives of the federal government. If fully developed, a 
govemmentwide strategic plan can potentially provide a cohesive 
perepective on the long-term goals of the federal government and provide a 
much-needed basis for fully integrating, rather than merely coordinating, a 
wide anay of federal activities. The development of a set of key national 
indicators could be used as a basis to inform the development of 
govemmentwide strategic and annual performance plans. The indicators 
could also link to and provide information to support outcome-oriented 
goals and objectives in agency-level strategic and annual performance 
plans. Successful strategic planning requires the involvement of key 
stakeholders. Thus, it could serve as a mechanism for building consensus. 
Further, it could provide a vehicle for the President to articulate long-term 
goals and a road map for achieving them. In addition, a strategic plan can 
provide a more comprehensive framework for considering organizational 
changes and making resource decisions. 


Concluding 

Observations 


The federal government is in a period of profound transition and faces an 
array of challenges and opportunities to enhance performance, ensure 
accountability, and position tlie nation for the future, In addition to the 
serious long-term fiscal challenges facing the nation, a number of 
overarching trends, such as defense and homeland security policies, 
increasing global interdependence, and advances in science and 
technology, drive the need to reconsider tlie proper role for the federal 
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government in the 21®* century, including what it does, how' it does it, who 
does it, and how it gets financed. This will mean bringing a variety of tools 
and approaches to bear. In our February 2005 report on 2F* century 
challenges, we outline a number of approaches that could facilitate a 
reexamination effort.®’ Today, I’ve discussed several of these, as well as 
some additional steps that I believe are necessary for an effective 
reexamination effort 

Much is at stake in the development of a collaborative performance 
budgeting process, niis is an opportune time for the executive branch and 
Congress to consider and discuss how agencies and committees can best 
take advantage of and leverage the new information and perspectives 
coining from the reform agenda under way in the executive branch. 
Through PMA and its related initiatives, including PART, the 
Administration has taken important steps in the right direction by calling 
attention to successes and needed improvements in federal management 
and performance. Some program improvements can come solely through 
executive branch action, but for PART to meet its full potential tlie 
asse^ments it generates must also be meaningful to and used by Congress 
and other stakeholders. 

Succe^ul integration of inherently separate but interrelated strategic 
planning and peiformance budgeting processes is predicated on 
(1) ensuring tiiat the growing supply of performance information is 
credible, useful, relink, and used (2) increasing the demand for this 
information by developing goals and measures relevant to the large and 
diverse community of stakeholders in the federal budget and planning 
processes, and (3) taking a comprehensive and crosscutting approach. It 
will only be through the continued attention of the executive branch and 
Congress that progress can be sustained and, more importantly, 
accelerated. This effort can both strengthen the budget process itself and 
provide a valuable tool to facilitate a fundamental reexamination of the 
base of government We recognize that this process \nll not be easy, Given 
the wide range of programs and issues covered, the process of rethinking 
the full range of federal government programs, policies, and activities could 
take a generation or more to complete, Regardless of the specific 
combination of reexamination approaches adopted, success will require 
not only the factors listed above but also sustained leadership throughout 
the many stages of the policy process. In addition, for comprehensive 
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reexamination of government programs and policies, clear and transparent 
processes for engaging the broader public in the debate are also needed. 

Mr. Chairman, this conclud«^ my prepared statement. I would be pleased to 
answer any questions you or the other Members of the Subcommittee may 
have at this time. 

For future information on this testimony, please contact Paul L. Posner at 
(202) 512-9573 or posnei^@gao.gov. Individuals making key contributions 
to this testimony include Jacqueline Nowicki, Tiffany Tanner, and Benjamin 
Ucht 
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I contend that agencies are better managed and achieving greater results today with 
the help of the President’s Management Agenda, but the opportunities for 
improvement are great. 

We want programs to work. We want to spend taxpayers’ money better every 
year. We want to make sure that the taxpayers get what they expect. 

One of our primary instruments for achieving this goal is the Program Assessment 
Rating Tool (PART). We use the PART to assess the performance of all Federal 
programs and to guide the action to improve their performance. 

With the PART, we are assessing programs to find out what works and what 
doesn’t. We ask of every program: 

• Does it have a clear definition of success, and is it designed to achieve it? 

• Are its goals sufficiently outcome-oriented and aggressive? 

• Is it well managed? 

• Does it achieve its goals? 
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In order for a program to be effective, it must have a clear definition of success and 
measures to determine whether it is achieving it. Each program assessed with the 
PAE.T is required to develop clear, outcome-oriented goals and targets for 
improving both performance and efficiency. PART analysis also helps identify a 
program’s strengths and weaknesses. In response to its PART assessment, a 
program identifies the specific steps it will take to improve its performance or 
overcome things that inhibit its performance. Under this model, all programs, both 
high and low performers, commit to improving each year. 

We have already begun to see success. As agencies have become better at 
demonstrating and focusing on results, PART ratings have improved. The 
percentage of programs rated Effective, Moderately Effective or Adequate rose 
from 57% in 2003 to 67% in 2005. The percentage of programs rated Ineffective 
or Results Not Demonstrated fell from 43% in 2003 to 33% in 2005. 

The Administration is committed to holding ourselves - agencies and programs — 
accountable to the American people for achieving results. One way we do this is 
through the transparency of the PART process. Currently, anyone can see the all 
completed PART questions and answers online at OMB’s website. We will also 
design a new website to more clearly communicate to the American people what 
programs are working, which ones are not, and what we are doing to make those 
programs better. 

We also need to involve Congress more directly in holding agencies and programs 
accountable for their performance through a Sunset Commission, which provides 
regular, formal scrutiny of Federal programs. This bipartisan commission would 
review each Federal program on a schedule established by the Congress to 
determine whether it is producing results and should continue to exist. Programs 
would automatically terminate according to the schedule unless the Congress took 
action to continue them. 

The Administration’s efforts to get more results for the American people are not 
only aimed at programs; they are behind the Administration’s effort to modernize 
the Federal Government’s personnel system. The Administration will soon 
propose legislation to, among other things, ensure employees are recognized and 
rewarded for their performance relative to mission-relevant goals, rather than 
longevity. It will require managers to ensure everyone clearly understands what is 
expected of them, how they are performing relative to those expectations, and how 
they can grow professionally and become even more effective each year. 
Continuous program performance improvement is possible with such personnel 
reforms. 
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Many programs don’t achieve their intended results because they are hampered by 
uncoordinated programs designed to achieve the same or similar goal. That is why 
the Administration proposes the enactment of Results Commissions, which would 
review Administration plans to consolidate or streamline programs that cross 
departmental or congressional committee jurisdictional lines to improve 
performance and increase efficiency. Ordinarily, programs that cross such 
boundaries often are not subject to the usual performance review process, resulting 
in inefficiencies, lost opportunities, or redundancies. Results Commissions, made 
up of experts in relevant fields, would be established as needed to review 
consolidation proposals. The Congress would consider the Commission’s 
recommendations through expedited review authority. 

The Administration has set a goal to reduce the deficit in half over the next five 
years and is working to stop growth in non-defense, non-homeland discretionary 
spending. In this context, it is even more imperative that we invest our resources 
in those programs that are performing well and those which hold the promise of 
performing well with reform. 

When we find that tax dollars can be invested with better result in another 
program, it is our responsibility to propose it. PART ratings of “Ineffective” or 
“Results Not Demonstrated” were a major factor in the decision to propose a 
number of reforms as well as the termination or reduction of 29 programs. For 
instance: 


HOPE VI - The program was originally designed to address 100,000 of the 
severely distressed public housing units in the Nation’s urban 
neighborhoods. Through 2004, 1 17,000 units have been demolished and 
HUD has approved the future demolitions of almost 50,000 more. The 
PART assessment found the program to be more costly than others and to 
take too long to produce results. So the budget redirects the funds other 
HUD programs. 


Juvenile Accountability Block Grants - Other than anecdotal information, 
there is little evidence the program reduces juvenile crime. The 
Administration proposes to redirect the program’s funds to other higher 
priority law enforcement programs. 
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Migrant and Seasonal Farm Worker Training Program - The PART 
assessment found that about 60 percent of participants receive no training 
and instead receive only low-cost supportive services that other Federal 
programs also finance. The Administration proposes to terminate the 
program, as it duplicates existing programs, does not focus sufficiently on 
job training, and has poor performance accountability for grantees. 

Just because we propose to terminate a program like the Safe and Drug Free 
Schools State Grants program doesn’t mean we don’t want safe and drug free 
schools. In fact, it is because we care so much about having safe and drug free 
schools, and independent evaluations show that the program doesn’t help us 
achieve that, we propose to invest the program’s dollars instead in a program that 
will hold grantees accountable for spending the money in areas with the greatest 
need on activities that have proven successful. 

We want programs to work. The PART helps us find out whether a program is 
working or not and, if not, what to do about it. In some cases, it may be that a 
program is such a low priority or performs so poorly that that program’s funds 
should be allocated elsewhere. It is our responsibility to convince Congress we are 
right. If we are successful, the result will be more programs achieving the intended 
results on behalf of the American people. 
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Mr. Chairman and Members of the Subcommittee: 

Thank you Chairman Cobum, Senator Carper, and Members of the Committee on 
Homeland Security and Governmental Affairs for inviting me to testify before you today 
on the present state of accountability and results in federal budgeting. This is an issue that 
we follow closely at the Government Accountability Project at the Mercatus Center at 
George Mason University. 1 should note the views expressed in my testimony are not an 
official position of the University. 

Much of my own research is concerned with the progress agencies have made towards 
developing outcome-oriented measures for their programs and results information, the 
effect the Performance Assessment and Rating Tool (PART) is having on budgetary 
decision-making and the extent to which Congress and the Executive use performance 
information in the budgetary process. I would like to submit to the record my 
forthcoming paper analyzing the results of the FY06 PART for your reference. 
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For the past several years our research team at the Mercatus Center has produced an 
Annual Scorecard that ranks agencies according to the quality of their annual 
performance reports, required under the Government Performance and Results Act 
(GPRA) (P.L. 103-62). 1 would like to submit our 2004 Annual Scorecard to the record 
for your reference. We have found that since GPRA’s passage, agencies have made slow 
but measurable progress towards the integration of budget and performance information 
by federal agencies. However, much work remains to be done. 

I. An overview: The importance of measuring performance 
Before considering how much progress has been made in implementing GPRA, 
PART and the relationship between the two, I’d like to explore the question: why are 
we doing this? Why should we bother to evaluate the performance of either agencies 
or programs, and what is the purpose of linking performance information with cost 
information? In focusing on performance are we taking a narrow approach, thinking 
only in terms of efficiency, rather than our government’s traditional broad focus on 
policy priorities that reflect a wide range of values? 

Policymakers can and do debate values and priorities. Such debates are a healthy 
aspect of democracy. But a debate about values and priorities is a debate about ends, 
not means. 

Policymakers often articulate many worthwhile ends. Most people of goodwill want 
to live in a society where the hungry are fed, the homeless are housed, dreaded 
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diseases are eradicated, and we are secure from terrorism (to name just a few.) If 
government is to accomplish these ends, rather than just make gestures towards 
accomplishing them, it is imperative that decision makers understand what means can 
most effectively accomplish those ends. 

This involves holding programs to a fact-based standard, not a values-based standard. 
If our goal is to reduce homelessness, does this program actually accomplish the 
goal? If so, to what degree does the program succeed? Do other, alternative, 
approaches reduce homelessness more effectively? How many fewer homeless people 
would we have if we moved resources from the programs that are less effective to 
those that are more effective? 

Answering these kinds of factual questions requires reliable performance and 
financial information. Linkage of performance and cost information tells us which 
means are most likely to accomplish whatever ends policymakers decide are 
worthwhile. 

II. OPRA 

GPRA is the most recent initiative of the past fifty years to promote the usage of 
performance budgets in the federal government to better inform budgetary decision- 
making. A performance budget can be defined as, “an integrated annual performance 
plan and annual budget that shows the relationship between funding levels and 
expected results. It indicates that a goal or set of goals should be achieved at a given 
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level of spending.”’ Performance budgeting means that money will be allocated not 
only based on policy priorities or perceived needs, but according to the government’s 
ability to address effectively those needs and priorities, ^ 

GPRA requires that agencies produce three types of reports: strategic plans, annual 
performance plans, and annual reports on program performance. In order to meet 
these reporting requirements, agencies must articulate their goals, design performance 
measures and assess results achieved.'’ This also requires that agencies be able to link 
their goals, objectives and performance measures with their budgets. 

One of three reports required by agencies under GPRA is an annual performance 
report, the purpose of which is to give the American people accurate, timely 
information to let them assess the extent to which agencies are producing tangible 
public benefits. In this way, GPRA has encouraged the development of performance 
measures and data. This information helps to assess the progress of agency activities 
towards meeting their goals. 

Since the GPRA requirements came into effect, the Mercatus Center at George 
Mason University has conducted an annual assessment of the 24 CFO agencies’ 
annual performance reports. The purpose of an annual performance report is to 
identify how much public benefit federal agencies produce for citizens, and at what 

‘ John Mercer, Performance Based Budgeiingfor Federal Agencies, AMS, Fairfax, 2002, p. 2 
^ Maurice McTigue, Henry Wray and Jerry Eliig, 5^ Annual Performance Report Scorecard: Which 
Federal Agencies Best Inform the Public? Mercatus Center at George Mason University, April 2005. 

^ McMurty, Virginia A., Performance Management and Budgeting in the Federal Government: Brief 
History and Recent Developments, CRS Report for Confess, December 5, 2003 . 
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cost. Mercatus’ Annual Scorecard evaluates the quality of these reports based on 
three main criteria: 1) how transparently an agency reports its successes and failures. 
2) how well an agency documents the tangible public benefits it claims to have 
produced; and 3) whether an agency demonstrates leadership that uses annual 
perfonnance information to devise strategies for improvement. 

In evaluating the second criterion: how well an agency documents tangible benefits to 
the public, agencies are assessed on four questions: 

• Are the goals and objectives stated as outcomes? 

• Are the performance measures valid indicators of the agency’s impact on its 
outcome goals? 

• Does the agency demonstrate that its actions have actually made a significant 
contribution towards its stated goals? 

• Did the agency link its goals and results to costs? 

It is the last of these four questions that enables us to assess how much progress agencies 
have made towards linking costs to performance. Knowledge of resource allocation and 
linkage to strategic goals, objectives, and performance measures are useful because they 
help agency management, Congress, and citizens understand what we are receiving in 
exchange for what we are paying. 

Strategic allocation of resources becomes possible only when financial information 
allows one to calculate the cost per unit of success and to compare alternative methods of 
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achieving the same goal. Where there are programs of equal efficacy, then the best means 
of comparing them is the cost per unit of success. This allows us to know how resources 
might be used to increase the public benefit. 

An agency cannot obtain the highest score on this criterion unless it breaks cost 
information down sufficiently to make such a calculation possible. The most meaningful 
linkage in this case is when an agency can link its performance measures to its costs. 

We rate agencies on a scale of 1 to 5 on this criterion. Each year, the criteria are tightened 
to reflect the previous year’s best practices. Roughly speaking, the following criteria 
determined scores for the fiscal 2004 reports: 

1 = no linkages to costs 

2 = costs linked only to strategic goals 

3 = costs linked to strategic objectives within strategic goals 

4 = costs linked to some level beyond strategic objectives 

5 = costs linked to individual performance measures 

In our earlier scorecards we found no agencies were truly meeting this criterion. It was 
not until our fourth annual scorecard in 2002 that any agency received the highest score. 
The Small Business Administration (SBA) received a score of five in that year because 
each performance indicator included a cost estimate. Some performance indicators 
included a cost per user or cost per output measure. In addition, the agency included a 
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chart that broke down costs for some activities. The agency, however, still lacked an 
overall listing of costs by strategic goals, though fhat could be easily calculated. 

SBA continued this best practice in 2003, receiving a score of five. SBA linked its FY 
2003 budget resources to its strategic goals and long-term objectives. The tables for each 
long-term objective further allocated costs down to each individual outcome performance 
measure under the applicable long-term objective. The cost information was presented 
going back to FY2000, making it possible to calculate each outcome. While the report 
was formatted to fully link costs to performance measures, the accuracy of the cost 
information is not certain given SBA’s financial management weakness. 

Agencies have made some general improvements in recent years, getting closer to linking 
costs with goals. In this year’s scorecard, seven agencies achieved a score of four on this 
criterion, the greatest number of agencies to achieve this score to date. 

These agencies: USAID, Commerce, Energy, Labor, SBA, State and Veterans Affairs 
broke down costs by more than just strategic goals and objectives. Commerce, USAID 
and Energy allocated costs among lower-level performance or program goals. The reason 
none of these agencies obtained the highest possible score is because they failed to 
allocate costs to individual performance measures. It is this level of detail that enables us 
to calculate what is paid for various types of successes. 
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Most agencies still lag far behind in this measure. For FY2004, five agencies received a 
score of one indicating they have no linkage of costs to goals or measures. However, 
there has been progress from our first scorecard in 1999, in which 14 agencies received 
the lowest score on this criterion. 

Many agencies are still at the stage of allocating budgets among strategic goals, too high 
a level for any meaningful performance-based budgeting. 

There are obstacles that agencies face in linking costs to performance data. Few agency 
financial and management information systems are capable of producing cost data linked 
to performance measures. Until better systems emerge, it is likely that agencies will 
continue to struggle to link costs to goals. 

In terms of the big picture of how agencies are performing in their annual reports, this 
year’s Annual Scorecard found that only 1 1 percent of the federal budget is covered by 
reports that averaged a score of 36 - the score a report would have if it received a 
“satisfactory” score of 3 on each of our 12 criteria. Based on their reports, it is difficult to 
tell what results are dollars are delivering. 
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in. How can the legislative and executive branches work together to forward the goals of 
GPRA? 


GPRA is unique among performance budgeting efforts in that it is a legislative and not an 
executive initiative. However, Congress has not paid much attention to the information in 
agency reports, though it requires them to be produced. When Congress does begin to use 
the information contained in agency reports, it will have the effect of motivating agencies 
to produce better results, better measures, and better data. This in turn will make 
performance information more accurate, reliable and valuable to Congress when 
allocating budgetary resources. Good information facilitates transparent and rational 
decision-making. 

The Bush Administration has attempted to use performance information in its budgetary 
decisions with the development of a tool to evaluate agencies’ programmatic 
performance. The Office of Management and Budget (0MB) created the Program 
Assessment Rating Tool (PART) in 2002 to evaluate all programmatic activity 
undertaken by agencies over a five-year period. To date, 60 percent of the budget has 
been rated, with another 20 percent currently under assessment. A program is evaluated 
in four areas: purpose and design, strategic planning, program management, and 
results/accountability. Based on a weighting of these criteria, a program can receive one 
of five ratings: effective, moderately effective, adequate, ineffective, and results not 
demonstrated. 
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PART is an attempt to take GPRA-iike analyses of results down to the level of individual 
programs. By formally linking budget requests with program performance, PART 
provides a view into how the Executive is making some of its budgetary decisions. 

The PART approach has several merits. It is a program-focused evaluation. This is a 
positive development if performance information is to influence budget decisions. Budget 
decisions are frequently made at the program level. When multiple programs aim to 
accomplish similar outcomes, a PART-like evaluation process can facilitate comparisons 
that help agency managers and Congress identify the most effective ways of 
accomplishing the agency’s goals. If we fail to make such comparisons, we guarantee 
that important needs will go unmet. While some may say this is inefficient, it is more 
accurately, irresponsible to those whose needs these programs are supposed to address. 

This does not mean that all of OMB’s PART evaluations are above criticism. We do have 
access to individual program questionnaires, but we may agree or disagree with 
individual PART evaluations. There remains a potentially subjective element to how 
ratings are assigned. The question format is limiting. There are legitimate difficulties that 
agencies face to relate PART’S performance assessment of programs to GPRA’s 
assessment of performance goals, which usually involve multiple programs. Given 
limited time and resources, it’s understandable that some agency managers may feel that 
PART directs their attention away from GPRA. And we understand that PART’S 
definition of performance or results, developed by 0MB in consultation with agencies, 
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may not always mesh perfectly with the performance goals or measures developed by 
agencies and stakeholders under GPRA. 

To the extent that the current PART evaluations fail to adequately assess program 
performance and provide a basis for comparison of similar programs, PART’S supporters 
and critics should both work to improve it. It would be a setback to the use of 
performance infonnation in budgeting, if criticisms or shortcomings of PART were to 
distract decision makers from the important task of systematically evaluating program 
performance. 

III. How has PART influenced Executive budget decisions ? 

In the “Major Savings and Reforms” in the President’s 2006 proposed budget, the 
Administration has recommended 1 54 programs for termination or reduction, 
According to this document, the Administration was guided by three principles in 
making these decisions 

1) Does the program meet the Nation’s priorities? 

2) Does the program meet the President’s principles for appropriate use of 
taxpayer resources? 

3) Does the program produce the results intended? 

Of these 1 54 programs, 99 were recommended for termination for a reduction of $8.8 
billion in spending. Of these 99 recommended terminations, 32 programs underwent a 
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PART review at least once, representing about $6 billion. Sixteen of these programs 
received a rating of results not demonstrated, six were rated adequate, and ten were rated 
ineffective. 

Of the 55 programs recommended for reductions in spending, 22 underwent a PART 
review at least once. Eight were rated adequate, four w'ere rated moderately effective, 
nine were rated results not demonstrated and one was rated ineffective. 

A little over one-third (or 54) of the programs recommended for either termination or 
reduction had been submitted to a PART review, representing about $10 billion in saving 
or 0.4% of the proposed $2,57 trillion budget. Within these 54 programs, PART appears 
to have been used in combination with other Administration criteria such as meeting the 
nation’s priorities and an assessment of whether the program is an appropriate use of 
taxpayer funds. That is, PART is not the only factor that is used to make funding 
decisions in the Executive, though it has advanced the use of performance information by 
linking program performance to the President’s budgetary proposals. 

In addition to reductions and terminations, PART also informed several program reform 
proposals in both the mandatory and discretionary categories. Notably, OMB performed a 
cross-cutting analysis of 35 Community and Economic Development programs across 
seven agencies in FY2005. As a result of this analysis, in combination with PART data 
on several of these programs, the Administration recommends the consolidation of 1 8 of 
these programs into a new initiative under the Commerce Department. 
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PART analyses are also responsible for recommended reforms to several mandatory 
programs. The Department of Treasury’s Financial Management (FMS) Debt Collection 
Initiative received an effective rating due to it being, “a well-established tool to collect 
delinquent non-tax debt.” This rating led the President to recommend in his 2005 budget, 
an initiative to increase opportunities to collect debt owed to agencies. 

There is not a perfect correlation between a program’s score and whether that program 
receives an increase or decrease in funding. That is, a poor score does not mean a 
program will necessarily be cut, and a high score does not mean a program will receive 
an increase. 


Looking only at those programs that were recommended for termination or funding cuts 
in the FY2006 Major Savings and Reforms, 25 of 179 programs rated results not 
demonstrated to date were recommended for cither termination or a reduction in funding. 
And half of the 22 programs rated ineffective to date were recommended for termination 
or cutting (several more ineffective programs were recommended for consolidation as 
part of budget reform proposals.) 
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Performance information is one of severa! factors that appear to have influenced the 
Administration’s proposed terminations, funding reductions, and reforms. 

PART is not the only means by which we may achieve better integration of performance 
information into the budget. Important arguments remain about PART’S mechanics and 
methodology, the extent to which the ratings are reflective of program performance and 
the issue of whether agencies are designing meaningful outcome measures as a result, or 
does PART create an exercise in minimum compliance. These criticisms should not 
detract from the fact that PART is a consistent and transparent attempt to evaluate 
program performance and to incorporate performance information into the budgetary 
process. It focuses on individual programs where budgetary decisions are often made. For 
this reason, PART is a valuable concept which can only improve the effort to advance 
performance budgeting. 

I would like to conclude by thanking you Mr. Chairman, Senator Carper, and Members of 
the Committee for taking on this important subject. The integration of performance 
information into the budget is a vital means of guaranteeing the stewardship of public 
funds will achieve the most effective results and ensure the greatest public benefits. I 
hope this testimony will be helpful to the Committee as it considers the role of 
performance information in the federal budgetary’ process. 
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TESTIMONY TO THE SENATE HOMELAND SECURITY AND GOVERNMENT 
AFFAIRS SUBCOMMITTEE ON FEDERAL FINANCIAL MANAGEMENT, 
GOVERNMENT INFORMATION, AND INTERNATIONAL SECURITY 
June 14, 2005 

BERYL A. RADIN, PhD 

PROFESSOR OF GOVERNMENT AND PUBLIC ADMINISTRATION 
UNIVERSITY OF BALTIMORE 


Chairman Cobum, Ranking Member Carper, and Members of the Senate 
Subcommittee on Federal Financial Management, Government Information, and 
International Security: My name is Beryl A. Radin and I am a professor of Government 
and Public Administration at the University of Baltimore and an elected fellow of the 
National Academy of Public Administration. I will be joining the faculty at American 
University’s School of Public Affairs this coming fall. 

For more than a decade, I have been studying the efforts within the federal 
government to improve the effectiveness and accountability of federal programs and have 
published a number of articles on this subject. I am currently completing a book 
manuscript that focuses on some of the unanticipated consequences of performance 
measurement activities. 

Like many others, I believe that it is important to find ways to assure that limited 
federal dollars are used effectively to carry out the goals and objectives of programs that 
have been created by both the Congress and the executive branch. Concern about 
performance attainment and performance measurement has spread beyond the public 
sector and it is hard to find any aspect of the American society today that does not focus 
on issues related to performance. 

But while the focus on performance is extremely important, I have serious 
questions about the current procedures that have been put in place to carry it out. While 
the effort that has been undertaken in 0MB through the Program Assessment Rating 
Tool (PART) may have been motivated by a legitimate and appealing concern, I do not 
think that this model is an appropriate way to measure program performance. Much of 
what has been devised in the name of accountability actually interferes with the 
responsibilities that individuals have to implement public programs. This includes PART 
as well as some other performance measurement efforts such as GPRA. 

The six issues that I will discuss today explain why there appears to be a 
disconnect between many of the PART evaluations overseen by 0MB and the budget 
proposals that were found in the President’s current budget. They illustrate how difficult 
it is to impose a single model on an extremely complex federal system with a diverse 
array of programs, 
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We should heed what H. L, Mencken once said: “Explanations exist: they have 
existed for all times, for there is always an easy solution to every human problem - neat, 
plausible, and wrong.” 

Let me summarize my six points. 

(1) Many federal programs have multiple and conflicting goals . 

The dynamics of the legislative process and the need to craft coalitions of support 
often create programs that try to accomplish several things at once (and they are 
not always compatible). The PART process does not reflect that reality and most 
of the evaluations assume that there is a single goal for a program. 

(2) Not all federal programs are alike . 

There are major differences between competitive grant programs, block grant 
programs, research efforts, regulatory programs and other program forms. Yet the 
PART approach largely treats them alike (even though OMB acknowledges the 
differences that have been analyzed by many others, including GAO). Some 
programs seek to expand opportunities for those who have not received the 
benefits of federal programs yet the PART process does not usually measure these 
goals nor does it seek information from those who represent the program 
beneficiaries. 

Political scientist James Q. Wilson has suggested there there are four different 
types of agencies; in some agencies outputs and outcomes can be observed and in 
others they cannot. He calls them production organizations, procedural 
organizations, craft organizations and coping organizations. He argues that in 
coping organizations outputs and outcomes are very difficult to measure. Yet 
PART does not really recognize these differences. 

Perhaps most importantly, the PART process does not recognize the decisions by 
Congress to enact programs in different forms. Instead, OMB actually second 
guesses Congress in terras of assessment of program purpose and design. This is 
most dramatically shown in the way that OMB has assessed programs that involve 
block grants to states. Congress decided in these programs to provide discretion 
to states and let them decide how to use the funds within often minimally defined 
parameters. This occurs because Congress in its wisdom has noted that problems 
are quite different in different states and a federally imposed measure is not 
appropriate in such a setting. 

When OMB rated block grant programs in its FY 2005 process, it found no block 
grant programs effective while finding 1 1% of programs examined that year rated 
effective. They also found 43% of the block grant programs to be ineffective 
while determining that only 5% of all the programs were ineffective. These 
ratings clearly suggest that the PART process is biased against block grant 
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programs. The rating of “ineffective” also ignores that programs have problems 
for different reasons. In some instances, the program does not have adequate 
staff or funding. In others, the federal government is involved in an issue but 
moves cautiously to address it because it is not clear how best to act. 


3) OMB budget examiners and 0MB itself have a limited perspective on 
programs . 

It does not make sense to rely on only one perspective to determine 
whether programs should live or die. Congress itself has recognized that as it has 
separated the authorizing and appropriating functions. A yearly budget process is 
not the only way to look at what are often very detailed and complex programs. 

In some cases, the PART ratings approved by OMB simply reflect historical 
views about programs that some budget examiners have held over many years. 

For example, it is not surprising that some of the health professions programs 
within HRSA in HHS receive low PART ratings since for many years OMB has 
recommended that these programs be defunded. Focusing only on the OMB 
perspective seems to have eliminated the possibility that stakeholder views will be 
included in the PART assessment process. 


(4) There are many different types of information that are useful to those who 
are charged with running or assessing programs . 

The information that is used in the PART process is not value neutral. Rather, it 
reflects markedly different reasons for a concern about performance. 

There are at least three agendas at play that are difficult to disentangle. Some 
advocates seek to eliminate programs and find it helpful to blame bureaucrats 
for problems. Others simply want to find a way to modify programs and argue 
that what worked in the past does not always make sense in a current or future 
environment. And still others believe that performance information will allow 
them to make a case for their programs and respond effectively with that data to 
those to whom they are accountable. PART does not allow a disentangling of 
these three agendas. 

Significantly, the information that is emphasized by OMB often is not always 
useful to program managers, policy planners, or evaluators or, judging from the 
quite tepid reaction on Capital Hill, to those charged with appropriations 
recommendations. 

(5) OMB calls for new data sources but does not acknowledge that agencies are 
not able to collect this data. 
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A number of agencies would like to collect data on the achievement of program 
outcomes. However, they are constrained both by the mandates of the 
Paperwork Reduction Act (which has required that they reduce the number of 
data elements collected) as well as by their inability to receive appropriations 
that would give them the resources to develop these data systems. In 
addition, there are sometimes limitations on data collection that are created by 
congressional decisions. For example. Congress has prohibited the 
Consumer Product Safety Commission from following cost/benefit analysis for 
some programs. Yet the PART evaluation criticized the Commission for failing 
to use cost-benefit analysis. 

(6) PART focuses on an executive branch perspective and is not easily transferred 
to the congressional branch. 

The one-size-fits-all approach that is found in the PART process is not compatible 
with a legislative branch with multiple committees and subcommittees as well as 
separation between authorizing and appropriations perspectives. The multiple 
venues within the Congress for discussing issues are one of the strongest 
attributes of our democracy even though the complexity it creates is sometimes 
frustrating. But that multiplicity itself makes it difficult to devise a single 
congressional perspective on performance. And as we know, legislation is 
constructed for a wide range of political reasons that may not be clear or relevant 
to 0MB budget examiners. Deferring to the executive branch and accepting 
PART wholesale cedes program effectiveness analysis to the executive branch. 

In conclusion, I suggest that this Subcommittee (and the Congress) avoid 
attempting to adopt the PART process and, instead, by focusing on accountability and 
results, emphasize the existing resources that are unique to the legislative branch. Instead 
of searching for a one-size-fits-all approach, the Congress has rich resources within the 
appropriations and authorizing committee structure that could be used to craft definitions 
of results within the framework of specific programs. Performance can best be handled 
within the confines of specific program development and traditional congressional 
oversight. 

The Congress has oversight capacity that can be used to provide more robust 
information than that from the PART process. The oversight process gives the Congress 
access to a range of information from GAO, CBO, CRS and the Inspectors General as 
well as from non-govemmental sources. Each of these sources has a somewhat different 
perspective but collectively they offer a rich view of program performance. Congress has 
the ability to develop a regularly scheduled assessment of programs within its oversight 
role. 


Thank you for inviting me to testify before this Subcommittee; I am available to 
work with the Subcommittee and its staff to continue this conversation. 
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Responses to Questions for Comptroller General Walker 
Government Accountability Office 
June 14. 2005 


Questions from Senator Carper 

Question 1. In its report from January 2004, "Performance Budgeting: 
Observations on the Use of OMB's Program Assessment Rating Tool for the 
Fiscal 2004 Budget', GAO stated that PART was not well integrated with GPRA. 
You concluded at that time that the 2004 PART process was a parallel, 
competing structure to the GPRA. You also noted that agencies were reporting 
that OMB's replacement of GPRA's strategic and performance goals with those 
of PART was detrimental to the GPRA reporting process and was a drain on 
resources. Furthermore, while OMB officials had expressed the need to reduce 
the number of GPRA performance measures, your report showed that overlaying 
PART requirements would actuaUy increase the total number of required 
measures, contrary to the goals of OMB officials. Your testimony didn't touch on 
any of these issues but, instead, portrayed PART and GPRA as complementary. 
Has GAO's position on the relationship between the two initiatives changed or 
have changes been made that make PART and GPRA reporting better 
integrated? 

We believe that PART should be better integrated with the broader planning process 
provided for in the Government Performance and Results Act of 1993 (GPRA).* As noted 
in my June IJ'" testimony, we remain concerned that, over time, as the performance 
measures developed for the annual PART process are used in the executive budget 
process, these measures may come to drive agencies’ strategic planning processes — a 
strategy contrary to GPRA’s notion of cascading strategic and annual plans. As part of 
the executive branch budget formulation process, PART must clearly serve the 
President’s interests. However, measures developed solely by the executive branch for 
the purposes of executive budget formulation may discourage their use in other 
processes, such as internal agency program management and the congressional 
authorization, budget, and oversight processes, especially if measures that serve these 
other processes are significantly revised or eliminated through the PART process. 

PART’S focus on outcome measures may ignore stakeholders’ needs for other types of 
measures, such as output and workload information. Like PART, GPRA states a 
preference for outcome measures and requires agencies to relate performance goals to 
resource needs. However, in practice, GPRA also recognizes the need to develop a range 
of measures — including output and process measures — to measure progress toward 
performance goals and related outcomes. Since different stakeholders have different 
needs and no one set of goals and measures can serve all purposes, PART can and should 
complement GPRA not replace it. 


Pub. L. No. 103*62 (1993) 
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GAO is continuing to monitor the implementation of PART, including its relationship to 
GPRA. At the request of the Chairracai of the House Subcommittee on Government 
Management, Finance, and Accountability, Committee on Government Reform, we are 
currently conducting a review of (1) OMB's and agencies' perspectives on the effects 
PART recommendations are having on agency operations and results and issues 
encountered in responding to PART recommendations; (2) OMB's leadership and 
direction in ensuring an integrated, complementary relationship between PART and 
GPRA, including how 0MB is assessing performance when multiple programs or 
agencies are involved in meeting goals and objectives; and (3) steps 0MB has taken to 
involve Congress in the PART process. We would be happy to provide you with a copy of 
this forthcoming report upon its issuance (expected in Fall 2005). 


Question 2. GAO has noted in the past that grant programs received lower than 
average PART scores. These low scores have often been a result of the 
programs' design, which allow for greater grantee decision making. Is there a 
danger of successful programs receiving low PART scores simply by operating as 
they were designed? If grant programs are more likely than other programs to 
receive lower PART scores, how should Congress use evaluations of grant 
programs when making funding decisions? Does PART need to be modified to 
better take into account the differences between programs? 

0MB has said that if a program manager’s ability to achieve results is hindered by an 
unclear purpose or suboptimal design, the manager should find it useful to have such 
obstacles highlighted. 0MB has also said that if statutory provisions impede 
effectiveness, one result of a PART review could be recommendations for legislative 
changes. It is therefore possible for a program to get a poor rating because it operates as 
designed. As you note, in our January 2004 report on PART,* we noted that one of the 
patterns 0MB identified in its ratings was that grant programs received lower-than- 
average ratings. To 0MB this suggested the need for greater effort by agencies to make 
grantees accountable for achieving overall program results. This need not eliminate 
grantee flexibility as to how to achieve results. At the same time grant structure and 
design play a role in how federal agencies are able to hold third parties responsible and 
can complicate the process of identifying the individual contributions of a federal 
program with multiple partners. Block grants present particular implementation 
challenges, especially when national goals are not compatible with state and local 
priorities. 

0MB has tailored the standard PART tool to try to account for different approaches to 
service delivery. In addition to the 26 questions among four broad topics that PART 
covers for all “programs” selected for review — (1) program purpose and design, (2) 
strategic planning, (3) program management, and (4) program results — PART also 
includes questions that are specific to one of seven mechanisms or approaches used to 


* GAO, Pertormance Budgeting: Observations on the Use of OMB's Program Assessment Rating Tool for 
the Fiscal Year 2004 Budget, GAO-04-174 (Washington, D.C.; Jan. 30, 2004). 
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deliver the program.® In addition, 0MB publishes both a single, bottom-line rating for 
PART results and individual section scores. 

Congress may find the individual PART section scores more useful than the overall 
summary score in conducting its oversight activities, because the complexity of federal 
programs makes it difficult to meaningfully interpret a bottom-line rating. For example, 
in the fiscal year 2004 PART, one program that was rated “adequate” overall got high 
scores for purpose (80 percent) and planning (100 percent), but poor scores in being able 
to show results (39 percent) and in program management (46 percent). In a case like this, 
the Individual section ratings provided a better imderstanding of areas needing 
improvement than the overall rating alone. Caution should be taken hi relying on bottom- 
line ratings for programs with multiple goals and when performance data are 
unavailable, the quality of those data is uneven, or they convey a mixed message on 
performance. 


Question 3. As you know, President Bush's Fiscal Year 2006 budget proposal 
targets about 150 programs for elimination or reduction. Of the programs that 
received OMB ratings, some scored poorly. Others scored rather well. In 
addition, I understand that there are even some programs that scored poorly 
that the President has asked us to preserve, in some cases even with increased 
funding. What does this tell you about how PART works? 

The relationship that you identified between PART scores and the President’s funding 
recommendations for fiscal year 2006 is similar to what we found for fiscal year 2004. In 
2004, we found that while PART scores were generally positively related to proposed 
funding changes in discretionary programs, the scores did not automatically determine 
funding changes. That is, for some programs rated “effective” or “moderately effective” 
OMB recommended funding decreases, while for several programs judged to be 
“ineffective” OMB recommended additional funding in the President’s budget request 
with which to implement changes,'' 

As we have previously stated, OMB and others recognize that performance is not the 
only factor in funding decisions.® Performance budgeting initiatives should not be 
expected to provide the answers to all resource allocation questions m some automatic 
or formula-driven process. Performance problems may well prompt budget cuts, 
program consolidations, or eliminations, but they may also inspire enhanced investments 
and reforms in program design and management if the program is deemed to be of 
sufficiently high priority to the nation. Conversely, even a program that is found to be 
exceeding its performance expectations can be a candidate for budgetary cuts if it is a 
lower priority than other competing claims in the process. The determination of 
priorities is a function of competing values and interests that may be informed by 
performance information but also reflects other factors, such as the overall budget 
situation, the state of the economy, security needs, equity considerations, unmet societal 
needs, and the appropriate role of the federal government in addressing any such needs. 


The seven major categories are competitive grants, block/formula grants, capital assets and service 
acquisition programs, credit programs, regulatory-based programs, direct federal programs and research 
and development programs. 

GAO, Pedonnance Budgeting: OMB's Pedormance Bating Tool Presents Oppodunities and Challenges 
tor Evaluating Program Performance. GAO-04-550T {Washington, D.C.: Mar. 11, 2004). 

’ GAO-04-550T, 5. 
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Questions for the Record from June 14, 2005 Hearing 
Before the Senate Subcommittee on 

Federal Financial Management, Government Information, and International Security 

Clay Johnson III 

Deputy Director for Management 
Office of Management and Budget 

1) During her testimony, Dr. Radin stated that the Program Assessment Rating Tool (PART) 
used to evaluate federal programs was unnecessary because Congress already had the resources 
and authority to effectively oversee federal programs. Do you believe that Congress has the 
resources, specifically the necessary amount of time, to effectively examine in detail the 
objectives and outcomes of every federal program, or is PART necessary to supplement the 
various efforts to conduct program oversight? 

I am not the right person to speculate on whether Congress has sufficient resources to 
effectively oversee federal programs. The PART can be a valuable resource for use in 
Congressional oversight of Federal programs. The Executive Branch is responsible for 
implementing programs and as such has access to more timely, detailed information about 
how all program perform and why some excel and other fall short. PART information can 
help the Congress target its limited resources to those areas that would beneflt most from 
additional Congressional oversight. The Executive and Legislative branches play 
complimentary roles in ensuring that programs achieve intended outcomes for the 
American people. Congress recognized that the Executive Branch would need to conduct 
the detailed, comprehensive program analysis when it passed the Government 
Performance and Results Act which requires the Executive Branch to evaluate the 
performance of its programs and report to Congress. 

2) Dr. Radin suggested during the course of the hearing that PART “is violating what the 
Constitution has created” by attempting to evaluate and rate the performance of federal 
programs. Do you believe that this is a valid critique of PART? Does PART circumvent the 
Congressional oversight process, or does it serve to supplement the process? 

PART does not violate the Constitution. Use of the PART reinforces the Congressional 
oversight process by providing the Congress with more detailed, comprehensive 
information. The PART is a tool the Executive Branch used to assess and improve the 
performance of its programs; it in no way limits Congressional authority. It is important 
to note that by law the Executive Branch is responsible for operating the programs that the 
Congress establishes. Most Americans would expect the Federal government to do a good 
job at implementing these programs without waiting for Congressional oversight to direct 
it to do so. 

3) One of Dr. Radin’s criticisms of PART was that it focuses too much on outcomes and 
efficiency values and not enough on process. Do you believe that programs can ultimately be 
improved without a sustained focus on goals and outcomes? 
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A focus on outcomes is essential to ensuring that programs achieve what was intended 
when they were created. One goal of a PART anafysis is to make sure that the strategies 
and methods we use achieve their intended goals. Sometimes an approach to solving a 
problem may not be as effective when the nature of the problem changes. The PART helps 
us make sure we are doing the right things to achieve the best results for the American 
people. It is important to note that Section 3 of the PART focuses on program 
management and assesses whether programs have the right processes in place to achieve 
their goals effectively and efficiently. 

4) Dr, Radin also stated during the hearing that the PART process “isn’t transparent[.]” Do you 
agree with this assessment? What measures have been or are being taken to increase the 
transparency of the program evaluation process? 

The PART process is extremely transparent as all information relating to the process is 
available on the internet. Each year the Office of Management and Budget publishes the 
list of programs that will be assessed, the schedule for assessing them, and the guidelines 
that will be used to assess them on the internet prior to beginning the assessments. 
Completed assessments are also published on the internet on www.ExpectMore.gov in both 
summary and detailed form. The detail includes an explanation and evidence for each 
answer to each question, the specific performance measures and data for each program, 
funding levels for the programs, and updates on what the programs are doing to improve. 
The launch ofExpectMore.gov was an important step in increasing transparency. While 
PART information has been publicly available each year since it began, this year with 
ExpectMore.gov the information was presented in clearer language that made it much 
more accessible to the public. 
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QUESTIONS AND RESPONSES FROM MS. RADIN 
Carper Questions for the Record. 

I. You note in your testimony that not all federal programs are 
alike, that many programs have multiple and conflicting goals and 
that PART doesn't reflect that. Why do you believe this is the case? 

Does PART do better or worse job than other, similar performance and 
budget integration initiatives? 


PART is too simplistic to handle the complexities in and differences between 
government programs. PART purports to ask questions targeted to each of 
seven broad types of programs; competitive grants, block/formula grants, 
regulatory programs, capital asset and acquisition programs, credit 
programs, R & D, and "direct federal programs." PART does not actually 
differentiate the questions it asks of each type of program. Instead, it is 
a one-size-fits-all questionnaire that takes the same basic approach to all 
of these divergent types of programs. 

Even if PART actually did a good job of tailoring its questionnaires to 
these seven types of programs, it still could well be overly simplistic. 

The PART approach assumes that there are generic activities that underly all 
credit programs or all regulatory programs, for example, and that there is 
neutral and objective information that adequately captures these generic 
activities. In fact, programs are quite distinct and have very different 
purposes, and it makes more sense to measure a given program's performance 
based on the specific expectations and purposes of a specific program. For 
example, OSHA and EPA's Clean Air Act regulatory programs are very 
different, serving very different purposes and statutory mandates. A 
simplistic list of generic questions could never yield as much valuable 
information for Congress or the public as a particularized inquiry specific 
to each program. 

My concern about PART compared to other performance budgeting systems is that it is 
much more centralized than other systems. Most other systems (PPBS, OPRA) actually 
begin with a bottom-up process. PART, by contrast, is much more of a top-down 
process. 


2. Are you aware of any instances in which PART has ignored the 
intent or will of Congress in rating programs? How much weight, if 
any at all, do you think OMB gives or should give programs success in 
carrying out the will of Congress or in meeting goals set out through 
planning programs like those set out through GPRA? 


There are many such instances. Here are just a few: 
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* The CPSC is instructed by Congress not to use cost-benefit analysis when 
issuing rules specifically required by law, such as the rules governing 
garage door openers and bicycle helmets. CPSC (which, despite an otherwise 
high passing score, was categorized "Results Not Demonstrated") was 
penalized for following the law and not conducting cost-benefit analyses for 
those rules. 

* CPSC was also scored down for not complying with OMB's tests for net 
benefits, even though CPSC's authorizing legislation instructs the agency to 
take a different approach in order to maximize public safety. 

* The same is true for OSHA and MSHA. OMB scored these programs negatively 
for failing to do "cost-benefit comparisons or monetiz[ing] human life," 

even though their organic acts and Supreme Court precedent forbid these 
practices. 

* OMB criticized the Appalachian Regional Commission (and flat-lined its 
budget request) for not being a "unique" program, because other existing 
agencies provide the same services. OMB completely misses the point of the 
Appalachian Regional Commission, which Congress created precisely because 
the existing patchwork of programs was failing to meet the needs of the 
extraordinarily impoverished population of that region. 

* Another program serving rural populations, HHS's Rural Health Activities 
program, was likewise penalized for following the very law that created it. 

OMB's criticism speaks for itself: "The major flaw of the Office's portfolio 
’‘stems from the programs' authorization’*" (emphasis added). The program was 
targeted for a drastic cut. 

* Every EPA research program PARTed to date was assessed as "Results Not 
Demonstrated," based on rationales that are deeply incompatible with the 
purposes of those programs. OMB criticized these programs for failing to 
link their research activities with the accomplishment of outcomes, but such 
criticism is willfully blind of the very nature and benefits of research: 

often we can learn as much from failure as from any success. 

OMB appears to give little or no credence to the intent of Congress. In 
fact, OMB deputy director Clay Johnson was asked directly during the 
hearing, "[I]s it possible for a program to get a poor rating simply because 
it does what's required by statute and not necessarily what OMB might like 
for that program to do?"-and Johnson replied, simply, "Yes." 

The very role of the executive branch in the constitutional order is to 
execute the law. It is inappropriate for OMB to be grading programs for 
following the law rather than OMB’s diktats. 
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3. As you know, President Bush’s Fiscal Year 2006 budget proposals 
targets about 150 programs for elimination or reduction. Of the 
programs that received 0MB ratings, some scored poorly. Others 
scored rather well. In addition, that there are even some programs 
that scores poorly that the President has asked us to preserve, in 
some cases even with increased funding. What does this tell you 
about how PART works? 

One thing it tells us is that there is no formulaic match between 
performance scores and budget requests. That much, at least, makes sense; 
low performance could mean either that a program needs more resources to do 
its job well or that a program is at the end of its life. (Of course, low 
performance scores could also mean that the 0MB assessor has simply 
concluded that the program is slated for the chopping block, no matter how 
well the program actually performs in meeting its statutory mandates.) 

Looking more deeply into the numbers beyond this initial level of generality 
reveals more disturbing patterns, however. 

* In the 2006 budget, programs rated "ineffective" apparently were targeted 
for elimination if they were housed in HUD or the Department of Education; 
of the programs that were rated ineffective and targeted for elimination, 78 
percent came from HUD or the Department of Education 

* Grant programs rate significantly lower in PART reviews than all other 
programs on average. Further, of the programs rated "ineffective" that were 
zeroed out completely in the 2006 budget, 89 percent were competitive or 
block/formula grants. 

Ultimately, the information about scoring and budget requests reveals the 
political nature of both the PART assessments and their linkage with budget 
requests. 


4. Ultimately, how would you recommend that Congress make decisions 
about which programs are effective and ineffective and which should 
continue or be eliminated? What should be ur standard for 
determining whether or not a program is a success and a good use of 
scarce resources? Do you think the questions OMB asks through the 
PART process are rigorous, detailed or unbiased enough to give us he 
information we would need to make informed decisions? 

The questions about which programs should continue or be eliminated are 
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inherently congressional questions. And Congress has the ability to use its existing 
resources through the authorizing and appropriations processes as well as the information 
available to it through GAO, CRS, CBO and the departmental Inspectors General. 

There is no simple answer that applies across the board to all programs to 
determine whether they are a success. There is no single definition of 
success that can be meaningfully applied to all government programs; what is 
meaningful is a particularized inquiry into whether a specific program is 
successful at achieving its specific statutory mandates. That success may 
not be captured in a single quantifiable measure; in perhaps most cases, the 
only meaningful assessment will come from gathering a large set of materials 
ranging from stakeholder assessments, both in and outside of government, to 
the program's history of engagement with its issues. The programs are much 
too complex to be reduced to a single number. 

OMB's measures are, as I have explained above, too simplistic to yield any 
useful information to assist Congress. 
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Executive Summary 


In February 2005, the Bush Administration released its proposed FY2006 budget marking 
the third year that the Office of Management and Budget (OMB) has evaluated 
government programs using its Program Assessment Rating Tool (PART). Designed as a 
means of encouraging agencies to develop performance measures and data in order to 
show program results, PART is used, in conjunction with other information, to make 
recommendations in the President’s budget proposals. This paper analyzes the 607 
programs rated to date by PART and seeks to determine how agencies have fared over 
time according to this methodology, the proportion of agency budgets rated ‘results not 
demonstrated’ or lacking in performance measures and data, and the relationship between 
funding levels in Congress and funding recommendations by the Executive to PART 
ratings. 

These findings show that while only a small part of overall total outlays, 6%, or $154 
billion are rated results not demonstrated, this represents, in some cases a large 
percentage of individual agency appropriations. This indicates that a large portion of 
some agency activities and budgets are unable to show measurable results according to 
OMB’s rating process. However, over the three year period that PART has been in place, 
there has been measurable progress by agencies to move from a results not demonstrated 
rating. This may indicate that agencies are responding to their assessments by improving 
their measurements and data, or that budget examiners are more experienced, or that a 
better set of programs is being evaluated each year. 

The PART methodology has been criticized for being a simplistic means of evaluating 
the often complex missions of various programs, though the application of the assessment 
process has brought a degree of transparency and consistency to the evaluation of federal 
programs. The information generated by the assessments has contributed to the 
advancement of performance budgeting in the Executive. 
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An Analysis of the Office of Management and Budget’s 
Program Assessment Rating Tool (PART)' 


In February 2003, the Bush Administration released with its proposed FY04 budget, a 
new method for evaluating the performance of federal programs called the Program 
Assessment Rating Tool (PART). PART is presented as an effort to get agencies to 
report consistently on their programmatic goals and results in order to facilitate funding 
decisions. It is one of five initiatives of the President’s Management Agenda.^ 

The evaluation of government agencies (also known as performance budgeting, or 
performance management) has been tackled by several previous administrations. PART 
is the first consistent methodology developed to evaluate federal programs. 

According to a Congressional Research Service report, PART can be viewed through two 
lenses.^ It has been argued that PART is a manifestation of the goals of the Government 
Performance and Results Act (GPRA). Passed in 1993, GPRA requires that agencies 
produce an annual report of programmatic progress and agency goals. By requiring 
agencies submit data on specific measures of program performance, advocates argue that 
PART has “breathed life into GPRA,” by holding agencies accountable for a program’s 
stated goals, through demonstrable measures of success. PART’S strength, it is argued is 
that it is a consistent, transparent and publicly accessible tool that has advanced the use of 
performance budgeting. 

Others have offered criticism of PART, notably a Government Accountability Office 
(GAO) report found after assessing the first year of PART data that there were some 
inconsistencies in what kinds of measures Office of Management and Budget (0MB) 
accepted as fulfilling program outcomes, and that the yes/no question format tended to 
result in an oversimplified assessment of program performance. GAO went on to note 
that 0MB has been open to such criticisms and scrutiny of its methodology. 

PART*,s Methodology 

PART requires that agencies submit an assessment of their programmatic performance to 
OMB over a six year period. To date, 0MB has rated 607 of lOOO’ programs it has 


‘ Prepared by Eileen C. Norcross, Research Fellow, Government Accountability Project, Mercatus Center 
at George Mason University. This paper is one in a series of working papers from the Mercatus Center’s 
Government Accountability Project and does not represent an official position of George Mason 
University. 

^ The five core management problems of the President's Management Agenda include; 1) strategic 
management of human capital 2) competitive sourcing 3) improved financial performance 4) expanded e- 
govemment 5) budget and performance integration, 

^ Brass, Clinton T., “The Bush Administration’s Program Assessment Rating Tool (PART)” CRS Report 
for Congress, November 5, 2004 

“ Government Accountability Office, “Performance Budgeting: Observations on the Use of OMB’s 
Program Assessment and Rating Too) for the Fiscal 2004 Budget (GAO-04-174)”, January 2004. 

’ Recently, OMB has spoken of revising the total number of programs to be assessed to 1200. 
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identified in the federal government. By FY08, all identified programs will have been 
assessed at least once. 

PART is designed as a series of between 25 and 30 Yes/No questions that are submitted to 
federal agencies for specific programs in a given fiscal year. There are four sections of 
the questionnaire — each weighted differently — dealing with an aspect of program 
performance: purpose and design (20%), strategic planning (10%), program management 
(20%) and results/accountability (50%). The individual assessments for each program are 
provided on OMB’s website.* 

part’s purpose is to combine performance and budgeting information in order to better 
inform the President’s budgetary recommendations. PART is also used as a means of 
addressing management and performance problems within programs. In addition to 
assessing information provided by the agencies, PART examines factors that may affect a 
program’s performance that may be beyond the agency’s control such as statutory 
provisions. 

A program may receive one of five ratings: ineffective, adequate, moderately effective, 
effective and results not demonstrated. The latter rating means that a program does not 
have enough information (either measures or data) to be rated, it does not imply the 
program is ineffective. 

Study Purpose and Previous Analyses 

With three years of PART data available, 1 undertook this analysis to see how agencies 
have been rated over time, how scores are related to Executive and Congressional 
appropriations, and what percentage of the federal budget is represented by particular 
program ratings. This study does not answer the question, “Is PART affecting agency or 
legislative behavior and funding decisions?” There have been two previous studies 
performed using PART data to answer this question using regression analysis, 

GAO undertook a regression analysis in 2004 using the first year of PART data in order 
to discover to what extent PART data had influenced the President’s FY04 budgetary 
proposal. By separating mandatory and discretionary programs, GAO assessed if PART 
scores influenced proposed funding changes between FY03 and FY04. The study found 
that PART scores have a positive and statistically significant effect on discretionary 
program funding levels in the President’s proposed budget, but there was no statistically 
significant relationship for mandatory programs. ’ 

A forthcoming regression analysis of PART to be published in the Public Administration 
Review by John B. Gilmour and David E. Lewis discovered that PART scores (for 2004 
and 2005) are positively correlated with the President’s recommendations.* 


’’ http://www.whitehouse.Bov/oinb/budget/fv2()06/p.ifi litml . (June, 2005) 
’GAO-04-174, p. 42. 

" Bra.ss, p. CRS-13 
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For the purposes of this analysis, I take PART’S ratings at face value. But, that does not 
mean I necessarily agree with the methodology used, or the conclusions arrived at in 
individual assessments. The President’s FY06 proposed budget includes a Major Savings 
and Reforms report that uses PART scores to make termination and funding decisions. I 
use this document to find descriptive evidence of how PART was used by the 
Administration in the FY06 proposed budget. Again, this does not imply an endorsement 
or criticism of how PART was applied to make these decisions. 

Key Findings 

The first section of the paper considers how programs have performed over time, and 
within program category and agency. 

Over time the number of programs rated results not demonstrated (not having sufficient 
information to evaluate performance), has declined from 50% to 29%; while the number 
of programs earning an effective rating have risen from 6% to 15%. This may be due to 
factors other than the influence of the PART assessment process. 

Among the 128 programs that have been reassessed, the number of results not 
demonstrated programs has fallen drastically from 78 to 8, while the number of programs 
rated adequate has increased from 18 to 50, and effective programs from 4 to 27. Only 
seven programs have retained their results not demonstrated rating over time. And only 
four programs fell in their ratings. Again, this may be due to a variety of factors. A 
drastic reduction in the number of programs rated results not demonstrated may indicate 
PART is pushing program managers to develop measures and data. 

Looking at the individual performance of agencies under the PART rating system, I 
found that five agencies with a high percentage of programs rated results not 
demonstrated also had the greatest percentage of their overall FY04 appropriations rated 
results not demonstrated indicating that programs OMB claim lack sufficient measures or 
sufficient data for evaluation not only consumed a large percentage of these agencies’ 
activities, but also a large percentage of their annual appropriations. The same 
relationship was tme for those agencies with a large percentage of programs rated 
effective. However, this relationship did not hold for every agency in every ratings 
category. 

I also examined the percentage of the budget (as total outlays), represented by certain 
ratings categories and discovered that while only 6% of total outlays ($134 billion) are 
rated results not demonstrated, some of these programs consume a large portion of a 
given agency’s appropriations. 

The last part of this analysis considers how funding decisions are related to PART scores. 
By comparing the Administration’s funding request in FY06 to what Congress 
appropriated to programs in FY05, I found that effective programs tended to be 
recommended for funding increases (61%) while ineffective tended to be recommended 
for funding decreases (86%). This does not mean that PART scores caused these funding 
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changes, or that a decrease represents a proposed cut or termination. Appropriation levels 
may change for other reasons such as expiration of a program’s authorization. 

In addition, I examined the President’s FY06 Major Savings and Reforms and 
discovered that of the 154 programs recommended for either termination or funding 
reduction, 55 had been PARTed at least once. Among these, 12 were rated ineffective, 
half of all PARTed programs rated ineffective.* This implies that a rating of ineffective 
does not guarantee a termination or reduction in funding. 

1 conclude the analysis by looking at the first two years of PART data and comparing 
what Congress appropriated in FY04 to what it appropriated in FY05 according to 
program ratings. As with the Administration, programs rated effective were more likely 
to receive funding increases (62%) and those rated ineffective were more likely to receive 
decreases (68%). Again, this does not mean PART played a role in all, or any of these 
decisions. 

How many programs are evaluated each year? 

in the FY04 proposed budget, the first year of PART, 234 programs were evaluated'®. 
However, several programs were removed from PART in ensuing years due to 
consolidation and eliminations bringing the actual total to 223. The following year, an 
additional 172 new programs were evaluated. In that year, PART reassessed 90 programs 
from FY04. Using the adjusted total, with the FY05 proposed budget a total of 395 
programs had been PARTed at least once. With the FY06 proposed budget, 212 new 
programs were assessed and 38 were reassessed. A total of 128 programs have been 
assessed more than once. By FY06, 607 programs had been PARTed at least once, 
representing 60% of federal programs". 


’ In addition to these 12, two other ineffective programs, Community Development Block Grant Formulas, 
and Rural Housing and Economic Development had both been recommended for de-funding as part of a 
reform initiative to consolidate these programs under the Commerce Department, bringing the total of 
ineffective programs recommended for cui.s or eliminations to 14. 

“ OMB states that 234 programs were PARTed in 2004. However, several programs were not included in 
ensuing PART reports: Center for Biologies Evaluation and Research, Center for Devices and Radiological 
Health, Center for Drug Evaluation and Research, Center for Food Safety and Applied Nutrition, Center for 
Veterinary Medicine were assessed as part of the Food and Drug Administration in the following year. 
Methane Hydrates, Geosciences Directorate, NSF Research Tools had their programmatic definitions 
change in ensuing years. Environmental Management was broken into two programs in 2004: R&D and 
Cleanup, but consolidated in 2005 and 2006. The Comanche Helicopter program is not mentioned in the 
2004 PART, though it was evaluated in that year. Several programs also underwent name changes between 
2004 and 2005, The Pension and Welfare Benefits Administration is referred to as the Employee Benefits 
Security Administration. Refugees to Israel is referred to as Humanitarian Migrants to Israel. Demining is 
referred to as Humanitarian Demining. 

" Four programs were assessed in 2005, but were not included in the 2006 PART: Aviation Passenger 
Screening, Disaster Relief Fund - Public Assistance, Support for Eastern European Democracy and 
Freedom Act, Supplemental Security Income for the Aged, bunging the most recent total of PARTed 
programs to 607 . 
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How PART has rated programs, cumulatively 

With 60% of federal government programs assessed, there has been a decrease in the 
cumulative number of programs rated results not demonstrated over the three year period. 
The cumulative number of programs rated effective has also increased. Ineffective 
programs dropped slightly from 5% to 4%, while moderately effective and adequate 
programs increased from 24% to 26% and 15% to 26%, respectively. This may imply that 
agencies have begun to develop performance measures in earnest, thus avoiding a results 
not demonstrated rating. The improvement in cumulative program results from year to 
year in other rating areas may be due to several factors: a) programs are improving their 
results information b) evaluations by 0MB are getting more accurate c) better- 
performing programs are being evaluated. 


Cumulative program 

results 2004-2006 

FY04 

FY05 

FY06 

Effective 

6% 

11% 

15% 

Moderately Effective 

24% 

26% 

26% 

Adequate 

15% 

20% 

26% 

Ineffective 

5% 

5% 

4% 

Results not Demonstrated 

50% 

38% 

29% 

Total 

234 

395 

607 


Cumulative program results by ratings category 
2002-2004 i^'^ 04-FY06t 



lyfeciive Moderaiely Adequate Ineffective Results not 
Effective ~ } Demonstrated 

Iq’IKH BMOy □20041 
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Are there observable changes in program performance between 2004 and 
2006? 


OMB has reassessed 128 programs out of the 607 assessed to date. Fifteen programs 
were evaluated three times. The remaining 1 13 were evaluated twice. 

The greatest improvement occurred with the decreased number of programs rating results 
not demonstrated. During the initial PART of the 128 programs, 76 were rated results not 
demonstrated. Upon reassessment only eight received that rating, seven programs 
retained their results not demonstrated rating, and one program fell from moderately 
effective to results not demonstrated. The number of programs rated effective also 
increased. Initially, four programs received the highest rating. Upon reassessment, 27 
programs were rated effective. Improvements also occurred in programs rated moderately 
effective (28 initially, 38 after reassessment) and adequate (18 initially, 50 after 
reassessment). There was an increase in the number of programs rated ineffective, from 
two to five, but the total number remains small. This raises the question of whether 
PART is motivating agencies to measure their performance or are other factors causing 
the improvement. 



Most reassessed programs did not reverse direction and slip in their ratings. Thirty-nine 
programs received the same ratings upon reassessment. Sixty-nine programs moved out 
of the results not demonstrated rating. Three of those were rated ineffective, the majority, 
36, earned an adequate rating upon reassessment. Four programs fell in their ratings: one 
moved from moderately effective to results not demonstrated (RND), and three programs 
fell from moderately effective to Adequate. 

For a detailed listing of these programs please refer to Table 1 in the Appendix. 
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Change in ratings fromfiist to nwst recent assessment 



Programs rated by program tvpe/categorv 

PART classifies programs according to seven categories: 

1) Block/Formula Grants - Programs that provide funds to State, local, and tribal 
governments and other entities by formula block grant. 

2) Capital Acquisition - Programs that achieve their goals through development and 
acquisition of capital assets (such as land, structures, equipment, and intellectual 
property) or the purchase of services (such as maintenance, and information 
technology). 

3) Competitive Grants - Programs that provide funds to State, local and tribal 
governments, organizations, individuals and other entities through a competitive 
process. 

4) Credit - Programs that provide support through loans, loan guarantees and direct 
credit. 

5) Direct Federal - Programs where services are provided primarily by employees of 
the Federal government. 
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6) Regulatory Based - Programs that accomplish their mission through rulemaking 
that implements, interprets or prescribes law or policy, or describes procedure or 
practice requirements. 

7) Research and Development - Programs that focus on knowledge creation or its 
application to the creation of systems, methods, materials, or technologies. 

Mixed programs are those that combine elements from two or more categories, (e.g. a 
Research and Development program that uses grants as a means of funding research). 

An analysis of PART data for FY04 through FY06 reveals that certain categories of 
programs fare better than others in the ratings. 


Cumulative ratings by program category 

^ RND ■ Ineffective □ Adequate □ Moderately Effective ■ Effective | 

R&D(8!) 

Regulatory Based (38 1 
Mixed (3 ) 

Direct Federal (188) 

Credit (24) 

Competitive Grants (112) 

Capital Acquisition (60) 

Btack/Formuia Grant (101 ) 

0% 20% 40% 60% 80% 100% 
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PART ratings by Program Category 


Program 

Category 

Results not 
Demonstrated 

Ineffective 

Adequate 

Moderately 

Effective 

Effective 

Block/Formula 

37% 

9% 

27% 

22% 

6% 

Grant (101) 

(37) 

(9) 

(27) 

(22) 

(6) 

Capital 

32% 

3% 

22% 

27% 

17% 

Acquisition (60) 

(19) 

(2) 

(13) 

(16) 

(10) 

Competitive 

42% 

4% 

31% 

18% 

5% 

Grants (112) 

(47) 

(4) 

(35) 

(20) 

(6) 

Credit (24) 

17% 

4% 

50% 

25% 

4% 


(4) 

(1) 

(12) 

(6) 

(1) 

Direct Federal 

25% 

2% 

26% 

28% 

19% 

(188) 

(47) 

(3) 

(49) 

(53) 

(36) 

Mixed (3) 

100% 

0% 

0% 

0% 

0% 


(3) 

(0) 

(0) 

(0) 

(0) 

Regulatory Based 

24% 

0% 

32% 

26% 

18% 

(38) 

(9) 

(0) 

(12) 

(10) 

(7) 

R&D (81) 

19% 

2% 

10% 

37% 

32% 


(15) 

(2) 

(8) 

(30) 

(26) 


Though mixed programs have the largest percentage of programs rated results not 
demonstrated at 100%, it should be noted that only three of these programs have been 
rated. 

Competitive Grants have 42% of programs rated results not demonstrated followed by 
Block Grants programs with 37% with results not demonstrated. Both Competitive 
Grants and Block Programs also have the largest percentage of programs rated 
ineffective, 4% and 9%, respectively. 

This information has lead OMB to undertake a cross-cutting analysis during FY06 of 
Block Grant programs. These programs are generally used to provide social services on 
the state and local levels. OMB notes that block grants pose performance measurement 
challenges because they are used for a wide range of activities and this difficulty is 
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reflected in its high number of programs rating results not demonstrated. The purpose 
of OMB’s analysis is to discover which programs are working best and apply those 
management methods to programs that are not meeting expectations. A PART 
reassessment will be performed after the recommendations are adopted. 

Research and Development programs, by contrast, have the highest percentage of 
programs rated Effective at 32%. Regulatory programs have the second highest 
percentage of effective programs with 18%, closely followed by Direct Federal programs 
with 19%, and Capital Acquisition programs with 17% rated effective. Six percent of 
Block Grant programs were rated effective and competitive grants and credit grants were 
rated with five and four percent effective, respectively. Nearly 50% of Block Grants, 
Capital Acquisition, Competitive Grants and R&D programs are rated either adequate or 
effective. Direct Federal and Regulatory programs received greater than 50% in a 
combination of these ratings. 

Programs rated by Agency'-^ 

When looking at the cumulative scores of agencies over the three years, some agencies 
have a relatively high number of programs that rate results not demonstrated. General 
Services Administration (GSA) comes in first with 67% of its programs unable to show 
results. 

GSA is closely followed by Education (63%), Veterans Affairs (57%), Department of 
Homeland Security (54%), Army Corp of Engineers (44%), Agriculture (41%), Interior 
(37%), Treasury and Housing and Urban Development (both with 30%), and Health and 
Human Services (26%). This rating again does not imply these programs are ineffective, 
but that there is not enough information available to judge their performance. 

Housing and Urban Development and the Department of Labor also have the largest 
percentage of programs that are rated ineffective: 20% and 19% respectively. 

To date, ten of the twenty programs rated by PART in Housing and Urban Development 
have either been unable to show results or are deemed ineffective. 


Office of Management and Budget, Budget of the U.S. Government fiscal year 2006 - Analytical 
Perspectives, p. 15 

’’ 0MB includes a category for smaller agencies called 'Other'. I have extracted five of the CFO agencies 
from this categorization for the analysis: Social Security Administration, General Services Administration, 
Nuclear Regulatory Commission, Office of Personnel Management and USAID. The remaining agencies in 
the Other category are: Consumer Product Safety Commission, Corporation for National and Community 
Service, Office of National Drug Control Policy, Export-Import Bank of the US, Tennessee Valley 
Authority, Federal Communications Commission, Federal Election Commission, Public Defender of the 
District of Columbia, Securities and Exchange Commission, Armed Forced Retirement Home, 
Broadcasting Board of Governors, Trade and Development Agency, American Battle Monuments 
Commission, International Assistance Programs, National Archives and Records Administration, 
Commodity Futures Trading Commission, Delta Regional Authority, National Credit Union 
Administration, Court Services and Offender Supervision Agency for the District, Neighborhood 
Reinvestment Corporation, Appalachian Regional Commission, and Denali Commission. 
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In terms of highest rated agencies, the National Science Foundation received ratings of 
effective for 100% of the seven programs that OMB has assessed to date. The Nuclear 
Regulatory Commission also had 100% of its three programs assessed to date rated 
effective. State was next with 63% of the 28 programs rated to date, assessed as effective. 
Defense had 43% of 23 programs rated effective. Treasury, NASA and Commerce all had 
at least of quarter of their assessed programs rated effective (35%, 31% and 26%, 
respectively). 


Agency 

Results not 
Demonstrated 

Ineffective 

Adequate 

Moderately 

Effective 

Effective 

Education (56) 

63% 

9% 

25% 

0% 

4% 


(35) 

(5) 

(14) 

(0) 

(2) 

Veterans Affairs 

57% 

0% 

14% 

29% 

0% 

(7) 

(4) 

(0) 

(1) 

(2) 

(0) 

Homeland 

52% 

0% 

18% 

18% 

12% 

Security (33) 

(17) 

(0) 

(6) 

(6) 

(4) 

Agriculture (56) 

41% 

0% 

23% 

32% 

4% 


(23) 

(0) 

(13) 

(18) 

(2) 

Other (33) 

45% 

0% 

21% 

27% 

6% 


(15) 

(0) 

(7) 

(9) 

(2) 


Interior (43) 

37% 

0% 

23% 

30% 

9% 


(16) 

(0) 

(10) 

(13) 

(4) 


Treasury (23) 

30% 

4% 

26% 

4% 

35% 


(7) 

(1) 

(6) 

(1) 

(8) 

Housing and 

30% 

20% 

20% 

30% 

0% 

Urban 

Development 

(20) 

(6) 

(4) 

(4) 

(6) 

(0) 

Health and 

26% 

3% 

38% 

23% 

9% 

Human Services 
(65) 

(17) 

(2) 

(25) 

(15) 

(6) 
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Agency 

Results not 

Ineffective 

Adequate 

Moderately 

Effective 


Demonstrated 



Effective 


NASA (13) 

23% 

0% 

15% 

31% 

31% 


(3) 

(0) 

(2) 

(4) 

(4) 

Justice (18) 

22% 

6% 

44% 

28% 

0% 


(4) 

(1) 

(8) 

(5) 

(0) 

EPA (32) 

19% 

6% 

66% 

9% 

0% 


(6) 

(2) 

(21) 

(3) 

(0) 

Commerce (23) 

17% 

0% 

26% 

30% 

26% 


(4) 

(0) 

(6) 

(7) 

(6) 

Defense (23) 

13% 

4% 

13% 

26% 

43% 


(3) 

(1) 

(3) 

(6) 

(10) 

Energy (43) 

12% 

5% 

14% 

53% 

16% 


(5) 

(2) 

(6) 

(23) 

(7) 

Labor (21) 

5% 

19% 

38% 

33% 

5% 


(1) 

(4) 

(8) 

(7) 

(1) 

State (27) 

0% 

0% 

15% 

22% 

63% 


(0) 

(0) 

(4) 

(6) 

(17) 

Transportation 

0% 

0% 

11% 

74% 

16% 

(19) 

(0) 

(0) 

(2) 

(14) 

(3) 

National Science 

0% 

0% 

0% 

0% 

100% 

Foundation (8) 

(0) 

(0) 

(0) 

(0) 

(8) 

Small Business 

14% 

0% 

43% 

29% 

14% 

Administration 

(7) 

(1) 

(0) 

(3) 

(2) 

(1) 

Social Security 

0% 

0% 

0% 

100% 

0% 

Administration 

(2) 

(0) 

(0) 

(0) 

(2) 

(0) 

General 

67% 

0% 

17% 

8% 

8% 

Services 

Administration 

(12) 

(8) 

(0) 

(2) 

(1) 

(1) 

Nuclear 

0% 

0% 

0% 

0% 

100% 

Regulatory- 
Commission (3) 

(0) 

(0) 

(0) 

(0) 

(3) 
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Agency 

Results not 
Demonstrated 

Ineffective 

Adequate 

Moderately 

Effective 

Effective 

Office of 

0% 

0% 

75% 

0% 

25% 

Personnel 

Managcment(4) 

(0) 

(0) 

(3) 

(0) 

(1) 

USAID (7) 

0% 

0% 

29% 

71% 

0% 


(0) 

(0) 

(2) 

(5) 

(0) 

US Army Corp 

44% 

0% 

21% 

27% 

6% 

of Engineers (9) 

(4) 

(0) 

(7) 

(9) 

(2) 


100% 

90% 

80% 

70% 

60% 

50% 

40^9 

30% 

20% 

10% 

0 % 
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Agency program ratings as a percent of agency FY04 appropriations 

It is more meaningful to ask what these program ratings represent in terms of their 
proportion to the agency’s total annual appropriation. The table below shows the ratio of 
the total of all FY04 appropriations of PARTed programs (grouped by rating) within an 
agency to the agency’s total FY04 appropriations received, according to their FY04 
Financial Statements. 
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Percentage of agency’s total Fy04 appropriations according to ratings category 


Agency 

Results not 
Demufitrat^ 

Ineffective 

Adequate 

Modo^tely 

Eifedive 

Effective 

Tot^ assessed as 
a percent of FY(M 
Agency 

Appropriations 

Total Agency 
FY04 

Appropriations 
Received ($mil) 

Agriculture 

18% 

0% 

13% 

46% 

0% 

78% 

$94,316 

Commerce 

4% 

0% 

36% 

17% 

22% 

80% 

$6,134 

Defense 

1% 

0.28% 

7% 

9% 

27% 

44% 

$582,010 

Education 

25% 

4% 

38% 

0% 

0% 

67% 

$72,090 

Energy 

!% 

Q% 

34% 

33% 

14% 

81% 

$24,190 

HHS 

2% 

0% 

2% 

44% 

4% 

52% 

$700,102 

DHS 

26% 

0% 

16% 

12% 

1% 

56% 

$38,303 

HUD 

5% 

23% 

0% 

41% 

0% 

69% 

$40,569 

DOJ 

4% 

0% 

18% 

20% 

0% 

42% 

$29,500 

DOL 

0% 

5% 

5% 

15% 

1% 

26% 

$58,039 

State 

0% 

0% 

6% 

4% 

40% 

51% 

$12,971 

Interior 

19% 

0% 

!2% 

5% 

3% 

39% 

$14,712 

Treasury 

1% 

0% 

1% 

0% 

1% 

3%. 

$352,212 

DOT 

0% 

0% 

16% 

707r 

8% 

94% 

$58,421 

VA 

63% 

0 

45% 

2% 

0 

110% 

$62,179 

EPA 

6% 

0% 

54% 

1% 

1% 

62% 

$8,353 

NASA 

29% 

0% 

4% 

28% 

20% 

82% 

$15,470 

SBA 

0% 

0% 

2% 

2% 

4% 

8% 

$4,430 

SSA 

0% 

0% 

0% 

16%' 

0% 

16% 

$699,906 

NRC 

0% 

0% 

0% 

0%' 

29% 

29% 

$593 

USAID 

0% 

0% 

15% 

0% 

n% 

32% 

$9,186 


For example, in FY04, USDA was given total appropriations of $94 billion. The total 
amount of money allocated to the 56 programs PARTed to date, in FY04 is $73 billion, 
representing 78% of its total FY04 appropriations. By grouping appropriations together 
based on rating, we can determine what portion of USDA’s appropriations fell Into a 
particular ratings category. USDA has 23 programs rated results not demonstrated, or 
41% of all USDA programs PARTed. Are these ail small budget programs, or do they 
consume a larger part of the agency’s appropriations? In this case, this represents 18% of 
total FY04 USDA appropriations. Fewer programs received a moderately effective rating 
(18, or 32% of PARTed USDA programs), but since one of these, the Food Stamp 
Program, is relatively large with $27 billion allocated in FY04, 46% of USDA's FY04 
appropriations are moderately effective. 

Sixty three percent of Veterans FY04 appropriations*** are rated results not demonstrated, 
followed by NASA with 29%, Homeland Security with 26% and Education with 25%. 
These agencies have the greatest portion of iheir annual appropriations rated results not 

According to Veterans Affairs Financial Statement for FY04, “Appropriations Received" amounted to 
$62,179 billion. This is le.ss than the $68,403 billion appropriated to PARTed programs in FY04, 
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demonstrated. In the case of the Education, 35 programs are rated results not 
demonstrated, representing 68% of PARTed Education programs, or 25% of the 
department’s FY04 appropriations. These programs range from the relatively small, such 
as the $9 million Occupational and Employment Information to IDEA Grants to the 
States, which was funded at $10 billion in FY04. 

In absolute terms, these agencies had a high percentage of their programs rated results not 
demonstrated, which also happens to represent a large percentage of their appropriations 
in FY04. 

Forty percent of State’s appropriations are rated effective (with 51% of its appropriations 
rated to date.) They are followed by Defense with 27%, Commerce with 22% and NASA 
with 20%. Again, this mirrors the percentage of programs rated effective. State and 
Defense had a large percentage of their programs rated effective (63% and 43%, 
respectively). 

Though Labor has a relatively high percentage of programs rated ineffective at 19%, this 
only represents 5% of its FY04 appropriations. HUD, however, has the highest number of 
programs rated ineffective at 20%, or four programs, and this mirrors how much of its 
FY04 appropriations are ineffective, at 23%, Two programs in particular account for this, 
Project-Based Rental Assistance ($4.7 billion allocated in FY04), and Community 
Development Block Grants ($4.3 billion allocated in FY04). Forty-one percent of HDD’s 
FY04 appropriations are rated moderately effective, (representing 6 programs or 30% of 
all PARTed HUD programs), and this is due largely to the performance of its Housing 
Vouchers program ($14.4 billion allocated in FY04). 

The majority of agencies’ programs, as a percent of appropriations, received either 
moderately effective or adequate ratings. 

What percentage of the budget is represented by PART ratines? 

The total amount of money appropriated in FY04 to the 607 PARTed programs was 
$1.25 trillion, representing nearly 60% of total FY04 outlays'"'’ (excluding interest on the 
debt). Breaking this out by ratings category, 6% of FY04 outlays are rated results not 
demonstrated. This represents $135 billion in FY04 appropriations. Though 6% of total 
outlays may seem relatively small, it should be noted that some agencies have a higher 
concentration of results not demonstrated programs than others, which consume a large 
part of their overall yearly appropriations. 


’’ For this calculation, 1 use the total amount of money appropriated in FY04 to all programs in a particular 
ratings category as my numerator. For the denominator, I have chosen to use Total Federal Outlays 
(excluding interest on the national debt) for FY04. This includes budgetary authority from previous years 
and is thus only a proxy for a ‘total federal appropriations figure’, which is not stated specifically in the 
budget, though it may be possible, to estimate this figure from the 13 individual appropriations bills. This 
calculation is therefore a rough estimate of what percentage of the budget is represented by certain rating.s. 
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For example, as noted earlier, Education has 63% (35) of its programs rated results not 
demonstrated or 25% of its FY04 appropriations ($17,8 billion). The majority of this is 
made up of one program: IDEA Grants to States ($10 billion), followed by Improving 
Teacher Quality State Grants ($2.9 billion). Federal Work Study ($999 million). 
Supplemental Educational Opportunity Grants ($770 million). Adult Education State 
Grants ($574 million), IDEA Grants for Infants and Families ($444 million), IDEA 
Preschool Grants ($388 million) and TRIO Student Support Services ($263 million). 

One percent of FY04 outlays are rated ineffective, representing $18 billion in FY04 
appropriations. Twelve percent of FY04 outlays are rated adequate and 10% are rated 
effective. Twenty nine percent of are rated moderately effective. 

The reason for such a large percentage of the budget being rated moderately effective is 
due to Medicare, which at $296.8 billion in FY04, represents half of the 29% of FY04 
outlays rated moderately effective. 


Percent of FY04 Outlays by PART rating 


Results not 
i-Deinnnsifjied 

j 6 ',: 


j inetteciive 

! r 1% 

! I 
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Total Amount of Money Appropriated in FY04 to PARTed Programs ($mil) 


Agency 

Results not 
Demonstrated 

Ineffective 

Adequate 

Moderately 

Effective 

Effective 

Total 

Agriculture 

$16,624 

$0 

$12,568 

$43,836 

$299 

$73,327 

Commerce 

$263 

$0 

$2,224 

$1,071 

$1,336 

$4,894 

Defense 

$5,793 

$1,620 

$40,947 

$50,951 

$154,265 

$253,576 

Education 

$17,852 

$2,359 

$27,351 

$0 

$182 

$47,744 

Energy 

$272 

$78 

$8,153 

$7,864 

$3,308 

$19,675 

HHS 

$12,160 

$2,188 

$15,360 

$310,559 

$27,160 

$367,427 

DHS 

$9,901 

$0 

$6,309 

$4,750 

$471 

$21,431 

HUD 

$1,996 

$9,274 

$160 

$16,652 

$0 

$28,082 

DOJ 

$1,077 

$59 

$5,364 

$5,887 

$0 

$12,387 

DOL 

$10 

$2,864 

$3,035 

$8,836 

$524 

$15,269 

State 

$0 

$0 

$827 

$552 

$5,176 

$6,555 

Interior 

$2,770 

$0 

$1,789 

$802 

$373 

$5,734 

Treasury 

$3,355 

$197 

$3,351 

$166 

$2,132 

$9,201 

DOT 

$0 

$0 

$9,444 

$40,920 

$4,666 

$55,030 

VA 

$39,214 

$0 

$28,207 

$982 

SO 

$68,403 

EPA 

$494 

$63 

$4,547 

$95 

$0 

$5,199 

NASA 

$4,529 

$0 

$696 

$4,385 

$3,074 

$12,684 

NSF 

$0 

$0 

$0 

$0 

$2,658 

$2,658 

SBA 

$0 

$0 

$78 

$93 

$169 

$340 

Other 

$18,656 

$0 

$91,659 

$127,859 

$2,678 

$240,852 

Total 

$134,966 

$18,702 

$262,069 

$626,260 

$208,471 

$1,250,468 


Has PART affected Executive budget decisions? 

How has the President used PART in making FY06 budget decisions? By considering the 
difference between the President’s funding request for FY06 and what Congress 
appropriated in FY05 to the 607 programs PARTed to date, there is a clear indication 
that, based on the most recent program rating, effective programs tended to be 
recommended for funding increases (61%), while ineffective programs had a higher 
percentage of programs recommended for funding decreases (86%). Programs receiving a 
results not demonstrated rating were recommended for more funding decreases (41%), 
than increases (30%). A greater percentage of programs rated moderately effective and 
adequate were recommended for funding increases (51% and 43%, respectively), than for 
decreases (38% and 36%, respectively.) 

This does not imply that PART is the only factor involved in these funding 
recommendations. There is no reason to expect a correlation between PART scores and 
individual funding decisions. An ineffective rating may mean a program is incapable of 
producing results, or that it requires managerial improvement. Programs may show a 
reduction in funding for a variety of reasons other than performance such as expiration of 
the program’s authorization. 


Government Accountability Project ♦ Mercatus Center ai George Mason University 


18 




106 


Based on change 
between 

President's FY06 
request and FY05 
actual 

appropriations 

Results not 
Demonstrated 

Moderately 

Effective 

Effective 

Ineffective 

Adequate 

Funding Increase 

30% 

51% 

61% 

5% 

43% 

No Change 

28% 

11% 

4% 

9% 

22% 

Funding Decrease 

41% 

38% 

35% 

86% 

36% 


Difference between President's FY 2006 funding request and FY 2005 Appropriation distributed by 
most recent program rating 

m% 

90 % 

80 % 

70 % 

60 % 

50 % 

40 % 

30 % 

20 % 

10% 

0 % 

Results not fAxJeraiety Effective Etfecove ineiiective Atlequate 

Demonstrated 



Q Funding increase 
■ NoCliange 
D Funding Decrease 


Descriptive evidence of how PART was used in the President’s 2006 
pro posed Budget 

Though PART was developed as a tool to help the Executive to make informed decisions 
about budget cuts, increases, and terminations, it was not the only factor used to make 
termination and reduction decisions in the 2006 proposed budget. In the FY06 proposed 
budget, the President included a Major Savings and Reforms document detailing which 
programs were recommended for termination, funding reductions, or reforms. In total, 
154 programs were recommended for either termination or funding cuts for a total 
savings of $17.2 billion 

Program terminations 

Of the 99 programs selected for termination ($8,8 billion in spending), 32 underwent a 
PART review at least once, these 32 programs represent about $6.4 billion in spending. 
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Sixteen of the programs recommended for termination received a rating of results not 
demonstrated, six were rated adequate and ten were rated ineffective. 



According to the Major Savings and Reforms, PART was an influence in many of the 
decisions to eliminate 32 of these programs, but the scores were used in conjunction with 
other information. 

Even Start, a family literacy program in the Department of Education, was rated 
ineffective. The Administration says it bases its recommendation for termination on both 
its PART scores and several independent evaluations of the program’s performance. 

As another example. Justice Department’s Juvenile Accountability and Block Grants 
Program was rated ineffective in 2004. The PART assessment states that evidence has not 
shown how it has contributed to a decreased juvenile crime rate in the last decade. 
Additionally, the 2006 budget says that the program competes with other priorities such 
as funds for Federal counterterrorism and reducing the Federal deficit. 

A detailed list of the 32 programs that have undergone a PART evaluation and, the reason 
given for termination are included in Table 2 of the Appendix. 

Program funding reductions 

Of the 55 programs recommended for budget reductions ($6.3 billion in spending), 23 
underwent a PART review at least once representing about $3.5 billion in spending. Two 
programs were rated twice: Interior Department’s Indian School Construction Program 
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(both times it was rated results not dentonstrated) and the EPA’s Clean Water State 
Revolving Fund, which moved from results not demonstrated in 2005 to adequate in 


2006. 


10 -| 

Ratings for PARTed programs suggested for budget cuts in FY 
'06 proposed budget 
(N=23) 
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Effective Demonstrated 


Again, PART information seems to have been used in conjunction with other evidence in 
making these decisions. 

The EPA’s Alaska Native Villages program, a PART assessment concluded, suffers from 
programmatic and financial weaknesses, “that prevent citizens from fully benefiting from 
the program.” It goes on to note that a State of Alaska financial audit uncovered, 
“unexplainable purchases of services and equipment, and poor project management that 
led to cost overruns and other wasteful spending.” The Administration recommends a 
reduction in funding from $30 million in 2005 to $15 million in 2006. The PART appears 
to have been used in making this decision along with information provided by an Alaska 
State audit. 

Health and Human Service’s Children’s Hospitals Graduate Medical Education Payment 
Program was rated adequate in 2005. According to the President’s proposed budget, a 
PART assessment concluded that the program doesn’t serve a real need since the 
financial condition of children’s hospitals is generally better than other hospitals, A 
reduction from $298 million in 2005 to $200 million in 2006 is requested in order to 
reduce funding, "... for a subsidy that has as its only purpose to provide funding to 
hospitals regardless of need or financial status.” 


Government Accountability Project ♦ Mercatus Center at George Mason University 


21 




109 


A reduction in funding from $7.05 billion to $6.5 billion was requested for Energy’s 
Environmental Management program which PART rated adequate in 2005 due to its 
successful completion of its timetable in the clean up of radioactive and hazardous waste 
sites at Federal facilities. The PART score seems to have played no role in this budgetary 
recommendation. 

Program Reforms 

In addition to reductions and terminations, PART also informed several program reform 
proposals in both the mandatory and discretionary categories. 0MB performed a cross- 
cutting analysis of 35 Community and Economic Development programs across seven 
agencies in FY05. As a result of this analysis, in combination with PART data on several 
of these programs, the Administration recommends the consolidation of 18 of these 
programs into a new initiative under the Commerce Department. 

PART analyses are also responsible for recommended reforms to several mandatory 
programs. The Department of Treasury’s Financial Management (FMS) Debt Collection 
Initiative received an effective rating due to it being, “a well-established tool to collect 
delinquent non-tax debt.” This rating led the President to recommend in his 2005 budget, 
an initiative to increase opportunities to collect debt owned to agencies. 

There is not a perfect correlation between a program’s score and whether that program 
receives an increase or decrease in funding. That is, a poor score does not mean a 
program will necessarily be cut, and a high score does not mean a program will receive 
an increase. 

Looking only at those programs that were recommended for termination or funding cuts 
in the FY06 Major Savings and Reforms, 25 of 179 programs rated results not 
demonstrated to date were recommended for either termination or a reduction in funding. 
And twelve of the 23 programs rated ineffective to date were recommended for 
termination or cutting (Two more ineffective programs were recommended for 
consolidation as part of a of budget reform proposal.) 








Rating 

Total 

Pn^rane 

PARTed 

Terminations of 
PARTed prt^r^ns 
(FY’06) 

Total Budget 
Savings ($000) 

Reductions in 
PARTed Programs 
(FY’06) 

Total Budget 
Saving (^WO) 

Results not 
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16 
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9 

$1,567 
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0 

$0 

4 

$604 
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22 

10 

$2,598 

2 

$286 

Effective 

90 

0 

$0 

0 

$0 

Adequate 

157 

6 

$755 

8 

$1,301 







Total 

607 

32 

$6,352 

2,? 

$3,758 
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How did Congress app ro priate money to PARTed programs (FYQ4-FY05)? 

By looking at the first two years of PARTed programs and comparing actual 
appropriations in FY04 to FY05 I was able to see how Congress appropriated money to 
these 395 programs. As with the Executive in its FY06 recommendations, a majority of 
programs rated ineffective were given funding decreases (68%). Similarly, a majority of 
effective programs (62%) were given funding increases. It is not possible to say to what 
extent (if any), PART scores influenced these decisions. 


Based on change 
in FY05 and 

FY04 actual 
appropriations 

Results not 
Demonstrated 

Moderately 

Effective 

Effective 

Ineffective 

Adequate 

Funding 

Increase 

51% 

60% 

62% 

26% 

60% 

No Change 

13% 

liq 

7% 

5% 

17% 

Funding 

Decrease 

37% 

30% 

29% 

68% 

23% 
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Conclusion 


With three years of PART data, some trends can be observed in how agencies and 
programs are faring in their ratings, how certain categories of programs tend to perform, 
and the extent to which ratings categories are related to Executive funding 
recommendations and Congressional funding decisions. The larger question of whether 
PART is having a measurable effect on funding decisions is not considered in this paper, 
though other scholarly assessments have examined this question for previous years by 
using regression techniques. Based on this study, we can observe relationships, but not 
infer causality. Certainly, PART is only one variable in making budgetary decisions, 
which are also based on political considerations, partisan priorities, legislative 
limitations, and the incentives present for legislators to retain or eliminate given 
programs. 

One conclusion we can draw is that in several cases, agencies with a large percentage of 
their programs rated results not demonstrated (or lacking in measures and data), also have 
a large percentage of their annual appropriations rated results not demonstrated. In some 
cases we may infer that large amounts of federal budgetary resources are spent on 
individual programs, or on a large portion of agency activity, but it is unknown if these 
resources are increasing the public benefit. 
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prior year balances are 
sufficient 

supported by other grant 
program 

limited impact; pvt sector 
can provide sves 

supported by other grant 
program 

2005 

Enacted 



1 

34.7 

oj 

20.8 

6.9 

2.2 

!2 

2.3 

2.5 

37.4 

Rating 













PARTed 

ou 

no 

no 

no 

0 

c 

no 

no 

ou 

no 

ou 

no 

ou 

Program 

Excellence in Economics 
Education 

Arts in Education 

Women’s Education Equity 
Grants 

Elementary and Secondary 
School Counseling 

Civic Education 

Star Schools 

Demonstration Projects to 
Ensure Quality Higher 
Education for Students with 
Disabilities 

Underground Railroad 
Program 

1 " 

Interest Subsidy Grants 

VR Migrant and Seasonal 
Farmworkers 

VR Recreational Programs 

Supported Eniploymenl 

Agency 
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Name in PART 



no performance 
standard 

L_ 









Reason for termination 

has sufficient pvt sector 
funding 

limited public benefits; pvt 
sector funding is sufficient 

noncompetitive program; 
same grantee for 30 years 

supported by other grant 
program 

supported by other grant 
program 

duplicative of Education 

Dept activities 

noncompetitive award 
system 

no evidence of impact or 
accountability 

no longer a need 

1 

limited 3 year project 

ineffective use of federal 
dollars 

2005 

Enacted 


fN 





09 

2 

59 


476 

Rating 












PARTed 

no 

ou 

o 

ou 

no 

no 

no 

no 

ou 

ou 

ou 

Program 

S 

£ 

£ 

% 

o 

o. 

o 

"S 

X 

Nuclear Energy Plant 
Optimization 

National Youth Sports 

Community Food and 
Nutrition 

Job Opportunities for Low- 
Income Individuals 

ACF Eaily Learning 
Opportunities Program 

CDC Congressional 

Earmarks 

CDC Preventive Health and 
Health Services Block 

Grant 

CDC Youth Media 

Campaign 

Direct Service Worker 

Grants 

HRSA Health Facilities 
Construction Congressional 
Earmarks 

Agency 

Energy 


HHS 
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Name in PART 










J 






Reason for terimnation 

lacks goals; no impact on 
expanding health care 
access 

lacks goals; no impact on 
expanding health insurance 
access 

duplicative 

duplicative 

served its purpose 

completed its goals 

earmarks 

duplicative 

unable to show impact 

unable to show impact 

duplicative 

no accountability for 
employment outcomes 

corporate subsidy 

utility of program is unclear 

2005 

Enacted 

oc 



o 

o 



o 

oo 

■O 

626 

ON 

CO 

50 

74 

NA 

ee 

c 

« 

ce 



J 









1 



PARTed 

ou 

ou 

O 

c 

no 

o 

c 

o 

c 

5 

o 

O 

ou 

ou 

0 

c 

ou 

ou 

Program 

HRSA Health Community 
Access Program 

HRSA State Planning Grant 
Program 

§ 

CJ 

« 

1 

«3 

H 

< 

X 

HRSA Universal Newborn 
Hearing Screening 

Real Choice Systems 

Change Grants 

BLM Jobs-in-the- Woods 
Programs 

NPS Statutory Aid 

Rural Fire Assistance 
Program 

Byrne Discretionary Grants 

Byrne Justice Assistance 
Grants 

National Drug Intelligence 
Center 

Reintegration of Youthful 
Offenders 

Maritime Administration 
National Defense Tank 

Vessel Contruction 

Federal Railroad 
Administration Railroad 
Rehabilitation Improvement 
Financing Loan 

e 

V 

OJ) 






Interior 



Justice 



1 

Transportation 
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Name in PART 











Reason for termination 

noncompetitive; earmarks 

Utility to applicants is 
limited 

independent evaluations 
conclude it is costly and 
likely to fail 

self-sufficient, consistent 
with authorization intent 

provides reimbursement to 
USPS; but no longer needs 
this funding 

not cost-effective 


ineffective use of federal 
dollars 

no longer a need 

some can be addressed in 
other programs; some 
cannot show results 

2005 

Enacted 

489 


291 

04 


<r) 


180 

137 

rr 

Rating 











PARTed 

ou 

no 

ou 

ou 

ou 

no 

ou 

ou 

ou 

some of 

these 

programs 

were 

PARted 

Program 

Unrequested Projects 

Water Quality Coo{^rative 
Agreements 

Hubble Space Telescope 
Robotic Servicing mission 

National Veterans Business 
Development Corporation 

Revenue Forgone 
Appropriation 

SBA Microloan Program 

SBA SBIC participating 
securities program 

Research and Exteniion 

Grant Earmarks and Low 
Priority Programs 

COPS Law Enforcement 
Technology Grants 

Other State/Local Law 
Enforcement Assistance 
Program Terminiations 

Agency 

< 


NASA 
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00 


Reason for reduction 

redirect funds to higher 
priority programs 

GAO recommends 
greater accountability 

can be provided by the 
slates 

pending review of best 
way to meet capacity 
requirements 

programs arc no longer 
cost-effective 

mission better carried 
out by Int'l agencies 

return program to core 
mission 

earmarks for 
noncompetitive grants 

countries have made 
progress 

consolidation 

activities can be 
commercialized 

concerns over technical 
complexity and costs 

change from 
2005 

-252 

o 

rs 

m 

P 

tS 

OO 


v-» 

r- 


o 


2006 

Proposed 

2327 


O 

o 

rM 

-144 

198 

r-1 

oc 

ts 

O 


o 

On 

m 


2005 

Enacted 

2579 

r- 

227 

189 

321 

0\ 

r- 

■<5' 


556 

ON 

569 

402 

Ratine 













PARTed 

ou 

OU 

no 

8 

o 

s 

ou 

o 

c 

o 

o 

c 

o 

s 

ou 

no 

Program 

Public Housing Capital 
Fund 

National Heritage Area 
Grants 

Payments in Lieu of 

Taxes 

Federal Bureau of 

Prisons Construction 
Program 

Juvenile Justice Law 
Enforcement Assistance 
Programs 

International Labor 

Affairs Bureaus 

Office of Disability 
Employment Policy 

Workforce Investment 

Act Pilots and 
Demonstrations 

Assistance for the 
Independent States of the 
former USSR 

Federal Railroad 
Administration - Next 
Generation High Speed 
Rail 

Aeronatics: Vehicle 
Systems Program 

Jupiter Icy Moons 

Orbiter 

Agency 




Justice 


Labor 



State 


NASA 
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Reason for reduction 

funds go to higher 
priority areas 

one-time appropriation 

PART noted 
weaknesses in 
incentives for 
controlling costs and 
allocating resources; 
need to improve 
accoutability for costs 
and measurement 

lower-priority program 
and duplicative 
authorized by 2002 

Farm Bill 

provides support 
through other programs 

lower -priority program 
and duplicative 
authorized by 2002 

Farm Bill 

new policy to phase out 
federal support for local 
planning after 20 yrs of 
funding; PART found it 
was duplicative 

low interest rates on 
loans mean more 
communities can afford 
to repay loans the 
program can operate at 
a higher loan to grant 
ratio. 

change from 
2005 


O' 

O' 

-653 

r- 

o 


-25 

-98 

2006 

Proposed 


o 

1444 

o 

o 

o 

Os 



450 

2005 

Enacted 


66 

2097 

fn 

00 

O 

sn 

00 

vs 

548 

Ratine 



RND 

Moderately 

Effective 

Adequate 

Q 

§ 

RND 

RND 

PARTed 

ou 

ou 

SJ 

ves 

V 


V 

ves 

Proeram 

National Historical 
Publications and Records 
Commission 

US Institute of Peace: 
Construction of New 
Building 

Forest Service Wildland 
Fire Management 

Environmental Quality 
Incentives Program 

Btoenergy Program 

Wildlife Habitat 

Incentives Program 

NRCS Resource 
Conservation and 
Development Program 

Water and Wastewater 
Grant.s and Loans 

Agency 






1 
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Reason for reduction 

Original legislation 
called for phase-out 
after 6 yrs.; less reliance 
on direct appropriations 
is needed 

PART found RND; 
work to improve 
program 

program is finishing its 
work 

PART found there is 
not a demonstrated need 

not a good use of 
federal funds: PART 
Score 

duplicative 

PART found it costly 
and slow 

PART found it was 

RND; no outcome 
measures in place 

both PART and the IG 
found monies were 
being misused 

PART found it was 
well-managed but 
recommends focusing 
on other activities 

change from 
2005 


-370 

O' 

in 

OC 

<7< 

so 


oo 

o 

o 

O' 

00 

O' 

<N 

2006 

Proposed 

47 

200 

6505 

002 


cn 

021 

522 

•sf 

(N 

2005 

Enacted 

108 

570 

7054 

298 

447 

r- 

238 

622 

263 


Rating 

Moderately 

Effective 

RND 

Adequate 

Adequate 

Ineffective 

Adequate 

RND 

RND 

RND 

Moderately 

Effective 

PARTed 

yes 



yes 


u 

>1 

yes 

yes 

yes 

yes 

Program 

Manufacturing Extension 
Partnership 

Adult Education Stale 
Grants 

Environmental 

Management 

HRSA Children’s 

Hospitals Graduate 
Medical Education 
Payment Program 

HRSA Health 

Professions 

■S 

u 

I 

3 

cx: 

< 

CO 

X 

Housing for Persons with 
Disabilities 

Native American 

Housing Block Grant 

Bureau of Indian Affairs 
School Construction 

US Geological Survey, 
Mineral Resources 

Program 

u 

s 

V 

sx 

< 

Commerce 

Education 

Energy 

HHS 



HUD 


Interior 
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